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CHAPTER 1 


Border Controls on Commercial Imports 


Performance audit reports 


This report presents the results of a performance audit conducted by the Office of 
the Auditor General of Canada under the authority of the Auditor General Act. 


A performance audit is an independent, objective, and systematic assessment 

of how well government is managing its activities, responsibilities, and resources. 
Audit topics are selected based on their significance. While the Office may 
comment on policy implementation in a performance audit, it does not comment 
on the merits of a policy. 


Performance audits are planned, performed, and reported in accordance with 
professional auditing standards and Office policies. They are conducted by 
qualified auditors who 


¢ establish audit objectives and criteria for the assessment of performance; 
¢ gather the evidence necessary to assess performance against the criteria; 
¢ report both positive and negative findings; 

* conclude against the established audit objectives; and 


* make recommendations for improvement when there are significant 
differences between criteria and assessed performance. 


Performance audits contribute to a public service that is ethical and effective 
and a government that is accountable to Parliament and Canadians. 


Table of Contents 
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Shared federal responsibility for controlling imports of consumer goods 
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Administering import requirements 


Most systems and practices for administering import requirements are well documented, 
but some need improvement 


Import requirements were administered correctly, with a few exceptions 


The Canada Border Services Agency’s framework to monitor administration of import 
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Targeting imported consumer goods 


Instructions are being followed for automated targets, with a few exceptions 
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What we examined 


Border Controls on Commercial 
Imports 


Main Points 


Under various acts and regulations, federal government organizations 
are responsible for ensuring the safety of commercially imported 
consumer goods that have been identified as a risk to the health and 
safety of Canadians. The Canada Border Services Agency (CBSA) is 
tasked with ensuring that commercially imported consumer goods 
enter Canada in conformity with applicable laws and regulations. 


To achieve this, the CBSA works with other federal government 
organizations to implement controls at the border. Controls at the 
border include administration of import requirements (for example, 
permits and authorizations), detection and interception of shipments 
targeted by federal organizations as high risk or non-compliant, 

and examination of selected shipments for admissibility into Canada. 
These controls are part of a broader product safety regime that also 
includes both pre- and post-border controls, such as licensing and 
market surveillance, and is supported by importers’ voluntary 
compliance. The regime is intended to control any high-risk products 
that are allowed to enter the country and thereby help ensure the 
safety of imported products in the marketplace. 


We examined the part of the product safety regime that is implemented 
at the border. Looking at selected commercially imported goods— 
including fertilizers, health products, pest control products, consumer 
products, consumer fireworks, vehicles, and tires—we examined how 
the CBSA, the Canadian Food Inspection Agency, Health Canada, 
Natural Resources Canada, and Transport Canada work together to 
ensure that products comply with applicable legislation when they 
enter Canada. 


Audit work for this chapter was substantially completed on 
30 September 2011. Further details on the conduct of the audit are 
in About the Audit at the end of this chapter. 
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Why it’s important 


What we found 


The volume of imports into Canada is so large that it is not practical 
for federal organizations to apply border controls to every shipment, 
nor is it necessary. According to the CBSA, it processed and released 
13 million shipments of commercial products in the 2010-11 fiscal year, 
about four million of which were subject to federal import requirements. 
Federal organizations must balance the requirement to examine 
shipments with facilitating the free flow of goods. To do this, they need 
to have systems and practices that can identify and concentrate on 
high-risk shipments of commercially imported products. 


Potentially unsafe products entering Canada without appropriate 
controls carry risks that could include illness and death. Even a less 
serious incident can have a significant effect on Canadian consumer 
confidence in imported products. 


- In most cases, imported consumer goods in our audit that pose a risk 
to the health and safety of Canadians are adequately controlled at 
the border by the Canada Border Services Agency, the Canadian 
Food Inspection Agency, Health Canada, Natural Resources 
Canada, and Transport Canada. With a few exceptions, the 
administration of import requirements and automatic targeting of 
high-risk shipments are working as intended. 


In the small percentage of cases where goods that did not meet 
import requirements were allowed to enter the country, most were 
products for which there was no agreement in place between Health 
Canada and the CBSA. While the CBSA has formal arrangements 
with the three other organizations in our audit, as yet it has no 
formal agreement with Health Canada that documents respective 
roles, responsibilities, policies, and procedures for implementing 
controls on several products under Health Canada’s responsibility, 
such as medical devices and pest control products. Until there is a 
formal agreement, border services officers do not have consistent 
instructions on procedures to follow for those products. 


Border lookouts and examinations are reserved for higher-risk 
shipments and they consume resources, yet their results are poorly 
documented. For example, examination results were recorded 
incorrectly or incompletely in 40 percent of cases we examined. 
Consequently, we could not determine whether the examinations 
are working as intended, nor can the organizations requesting them. 
Moreover, among the audited organizations there are gaps in the 
monitoring of all three border controls. For example, the CBSA’s 
target monitoring reports are inaccurate and incomplete. These gaps 
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make it difficult for federal organizations to know how well the 
controls are working and where resources and effort can be directed 
most effectively to manage risk. 


The entities have responded. The entities agree with all of 
the recommendations. Their detailed responses follow the 
recommendations throughout the chapter. 
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BORDER CONTROLS ON COMMERCIAL IMPORTS 


Introduction 


1.1 Each year, millions of imported products cross Canada’s borders, 
destined for the Canadian marketplace. Many commercial products 
must meet specific requirements defined in federal legislation before 
they can be imported into Canada, especially those that have been 
identified as a risk to the health and safety of Canadians. Various 
federal organizations, such as the Canadian Food Inspection Agency 
(CFIA), Health Canada, Natural Resources Canada, and Transport 
Canada, administer this legislation. They have established product 
safety controls before goods are imported, during the importing 
process, and after products have entered the country. This regime is 
intended to control any high-risk products that are allowed to enter 
the country and thereby help ensure the safety of imported products in 
the marketplace. 


1.2 Without appropriate controls, some products could enter 
Canada that may present serious risks to the health and safety of 
Canadians. Even a less serious incident can have a significant effect 
on Canadian consumer confidence in imported products. Imported 
consumer goods have attracted more attention since 2007, when 
contaminated toothpaste and toys with lead paint and magnetic 
parts that could cause choking entered the country. As a result, 

in December 2007, the Prime Minister announced the Food and 
Consumer Safety Action Plan. The plan provided additional resources 
to Health Canada, the CFIA, the Public Health Agency of Canada, 
and the Canadian Institutes of Health Research to improve controls 
at all stages of the product safety regime. The plan focused on the 
following three key areas: 


¢ Active prevention initiatives included enhanced guidance for 
industry on regulatory requirements, steeper fines and penalties, 
development of safety standards, and more accessible consumer 
safety information. 


* Targeted oversight initiatives included enhanced licensing and 
compliance verification, improved surveillance, and mandatory 
reporting of adverse events and incidents. 


¢ Rapid response initiatives included risk communication to federal 
organizations and the public and the power to recall products 
from the marketplace. 


1.3. The three border controls we examined in this audit— 
administration of import requirements, targeting of high-risk shipments, 
and examinations—support targeted oversight and rapid response. 
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Release—A decision by the Canada Border 
Services Agency to allow goods to leave customs 
control and enter the Canadian marketplace. 
This may occur at border points of entry or at 
inland customs offices. 


Shared federal responsibility for controlling imports of consumer goods 


14 The Canada Border Services Agency (CBSA) is required to 
ensure that commercially imported consumer goods enter Canada in 
conformity with applicable legislation. The Customs Act requires that 
all imported goods be reported to the CBSA and makes the Agency 
responsible for determining whether to release the goods into Canada. 
Border services officers have the power to examine, detain, or seize 
imported goods if they suspect non-compliance with legislation, 
including laws under the jurisdiction of other federal organizations. 
All these other federal organizations must therefore collaborate with 
the CBSA to ensure that goods entering Canada meet the 
requirements of the legislation they administer. 


15 Inthe 2010-11 fiscal year, the CBSA processed 13 million 
shipments of commercially imported products. In addition to being 
subject to the requirements of the Customs Act, the CBSA estimates 
about four million of these commercial import transactions also fell 
under one or more other federal laws or regulations. Federal 
organizations cannot examine every shipment. They must balance the 
requirement to examine shipments with facilitating the free flow of 
goods. To do this, federal organizations need to have systems and 
practices that can identify and concentrate on high-risk shipments of 
commercially imported products. 


16 Inthe 2010-11 fiscal year, the five audited organizations 
reported they spent approximately $200 million to administer program 
activities intended to ensure the safety of the imported products 
included in the scope of the audit. This amount includes the border 
controls examined by the audit. 


1.7 Exhibit 1.1 summarizes the goods included in the scope of our 
audit, the laws that regulate them, and the four federal organizations 
responsible for controlling the importation of these goods. The exhibit 
also summarizes the import requirements, the requirements 
administered by CBSA officers, and the risks to Canadians if the goods 
do not meet these requirements. 
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Exhibit 1.1 Import requirements for goods included in the audit 


Canadian Food Inspection 
Agency (CFIA) 


Fertilizers Act and associated 
regulations 


The CFIA must ensure that 
imported regulated fertilizer 
and supplement products are 
safe for humans, piants, 
animals, and the environment; 
effective for their intended 
purpose; and properly 
labelled. 


Fertilizers and supplement products — 


Border Services Agency 


ior required at the border by the. & 


Risks of non-compliance with 
requirements 


Import requirements vary by 
nature of the product, country 
of origin, and use of the 
product. These could include 
registering products, obtaining 
permits, and adhering to 
specific packaging and 
labelling requirements. Some 
low-risk products are exempt 
from import requirements. 


Border services officers are 
expected to review the 
shipment’s supporting 
documentation and the 
request for release. Low-risk 
imports do not require any 
action by border services 
officers. 


A toxic fertilizer or supplement 
could have an immediate 

or long-term harmful effect on 
the environment, might ’ 
endanger the environment on 
which human life depends, or 
might endanger human life or 
health. 


Health Canada Health products (including drugs, natural health products, and medical devices) 


Food and Drugs Act, the 
Controlled Drugs and 
Substances Act, the Pest 
Control Products Act, the 
Canada Consumer Products 
Safety Act, the Hazardous 
Products Act, and their 
associated regulations 


Health Canada is responsible 
for protecting the public by 
addressing or preventing risks 
to human health or safety that 
imported products pose. 


Import requirements vary by 
product, but may include 
market authorization 
(human-use drugs, natural 
health products, some 
medical devices), obtaining 
permits, site or establishment 
licensing (human-use drugs, 
natural health products, some 
medical devices), and 
adhering to packaging and 
labelling requirements. 


Border services officers are 
expected to contact Health 
Canada if they have a 
compliance concern or need 
more information. However, if 
the shipment contains a 
controlled substance, the 
officer must verify the import - 
permit against the quantity for 
each individual shipment, and 
ensure that the importer is 
licensed. 


s 


Unauthorized health products 
may be ineffective, unsafe, 
and of poor quality. 

For example, severe side 
effects might occur if 
Canadians take an 
unauthorized drug or natural 
health product. 


Regulated consumer products (toys, child care items, cribs, cradles, strollers, consumer 


chemicals, and cosmetics) 


Import requirements include 
labelling and meeting safety 
and performance standards. 
Cosmetic preparations are 
also subject to the Food and 
Drugs Act and its regulations 
regarding composition, safety, 
labelling, and advertising. 


Border services officers are 
expected to contact Health 
Canada if they have 
compliance concerns or need 
more information. 


7 


Consumer products that do 
not meet Canadian regulations 
could cause injury or death. 


Unregulated consumer products 


There is no import 
requirement for unregulated 
consumer products. 


| Borde services officers have 


no requirements concerning 
these products unless Health 
Canada has identified 

a product as high risk and 
asked the Agency to target it 
for inspection. Officers are 
then to follow instructions that 
Health Canada provides. 


Consumer products, such as 
ski helmets, that do not meet 
Canadian standards could 
cause injury or death. 
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Exhibit 1.1 Import requirements for goods included in the audit (continued) 


Federal organizations responsible 
for imported goods and 
their relevant legislation 


Health Canada (continued) 


Import requirements 


Pest control products 


Canada Border Services Agency 


Actions required at the border by the 


Products must bear 

the approved label for sale in 
Canada, and an import 
declaration form must be 
included at the time of 
release. 


Border services officers are 
expected to contact Health 
Canada if they have a 
compliance concern or need 
more information. 


Natural Resources Canada 
(NRCan) 


Explosives Act and associated 
regulations 


NRCan is responsible for 
ensuring that only fireworks 
that are safe and authorized 
under the explosives 
regulations are imported into 
Canada. 


Transport Canada 
Motor Vehicle Safety Act 


Transport Canada regulates 
the importing of vehicles and 
vehicle products to reduce the 
risk of death, injury, and 
damage to property and the 
environment. 


Consumer fireworks 


An unregistered pest control 
product may be unsafe for any 
use, and often does not 
contain precautions or 
instructions in either of 
Canada’s official languages. 
This could lead to accidental - 
misuse or injury. 


The product must be on the 
list of authorized explosives 
and have an import permit. 


Border services officers are 
expected to validate the 
Explosives Import Permit. If 
the shipment is not as 
described in the permit or the 
goods are not accompanied by 
a permit, officers are to detain 
the explosives and refer them 
to NRCan for review. 


Vehicles 


Unauthorized fireworks could 
result in injury, property 
damage, or death. 


Imports must be accompanied 
by a vehicle import form, 

a statement of compliance 
label, a vehicle identification 
number, a certificate of title, 
and manufacturer’s certificate 
of origin or salvage title. 


Tires 


Border services officers are 
expected to ensure that 
vehicles meet all import 
requirements, that the vehicle 
form is completed correctly, 
and that they stamp the form. 


Foreign manufacturers must 
design and build vehicles that 
meet standards and 
regulations under Canada’s 
Motor Vehicle Safety Act. 
Vehicles that do not meet 
standards increase the risk of 
death, injury, and damage 

to property and 

the environment. 


On-road new tires must be 
accompanied by a declaration 
that the tire complies with 
Canadian Motor Vehicle Tire 
Safety Regulations. On-road 
used tires require a United 
States Department of 
Transportation (DOT) symbol 
on the tire and the 
identification number. 


Off-road used tires must 
display an “Off-road use only” 
designation. 


Border services officers are 
expected to verify the 
importer’s declaration of 
compliance, and the DOT 
symbol (on-road used) or off- 
road use designation. 


SAS RSE 


On-road tires must meet 
Canadian or United States 
standards. Tires that do not 
meet standards increase the 
risk of a crash or injury. 
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Target—An automated notice that ACROSS, the 
Canada Border Services Agency’s importing 
information system, issues whenever an 
importer’s business number or other data 
triggers an alert. Officers must analyze and 
follow instructions accompanying the target. 
Targets can also be issued via email from Agency 
headquarters directing officers to take specific 
action regarding shipments. 


ACROSS—The Accelerated Commercial Release 
Operations Support System of the Canada Border 
Services Agency, which enables importers and 
brokers to transmit information about their 
imports electronically. A border services officer 
reviews the information, decides whether to 
release the shipment, and sends the response 
back to the importer or broker electronically. 


Referral—A request from the Canada Border 
Services Agency to an appropriate federal 
organization to validate the admissibility of a 
shipment based on legislative requirements. 
Referrals are communicated by phone, facsimile, 
or email. 


BORDER CONTROLS ON COMMERCIAL IMPORTS 


Border control processes 


1.8 Federal organizations responsible for ensuring that imported 
goods meet all Canadian requirements also rely on importers’ voluntary 
compliance with federal legislation. Commercial clients (importers, 
brokers, carriers, and freight forwarders) report the goods they want to 
import by submitting detailed information about the goods and the 
importer before or upon arrival at the border. Commercial clients are 
required by legislation to provide accurate and complete information. 
Border services officers review this information to decide whether to 
release the goods into Canada. 


19 Federal organizations may request that the CBSA target and 
intercept shipments for examination at the border. To determine 
whether a shipment represents a high risk and should be detained for 
examination, border services officers use these targets, automated 
systems, their own judgment, and advice from other federal 
organizations. If an officer decides that an examination is not required, 
the Agency releases the goods. If the officer decides an examination is 
required, the officer directs the goods to an examination facility, where 
the goods are released only if they pass the examination. 


1.10 The CBSA may refer information about the shipment to the 
appropriate federal organization for document review; or it may 
conduct a physical examination on behalf of the appropriate federal 
organization; or an inspector from the appropriate federal organization 
may conduct a physical examination. Commercial shipments can be 
examined at a facility at the port of entry or, in certain cases, inland. 


Focus of the audit 


111 The objective of the audit was to determine whether the Canada 
Border Services Agency (CBSA), the Canadian Food Inspection 
Agency (CFIA), Health Canada, Natural Resources Canada 
(NRCan), and Transport Canada control, at the border, selected 
commercially imported products that pose a risk to the health and 
safety of Canadians. 


112 We examined the product safety regime that is implemented at 
the border. We did not audit, and therefore cannot comment on, the 
effectiveness of pre- and post-border controls or voluntary compliance. 
Controls before goods are imported and after they have entered the 
country have been the subject of other audits: most recently, in the 
2011 Fall Report of the Auditor General, Chapter 4, Regulating 
Pharmaceutical Drugs—Health Canada, and the 2011 June Report of 
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Trusted trader programs—Three Canada 
Border Services Agency programs (Free and 
Secure Trade, Partners in Protection, and 
Customs Self Assessment) that designate 
certain companies as trusted and low risk 
because they have met certain system and risk 
requirements and are subject to Agency audit. 


Administering import requirements 


the Auditor General, Chapter 6, Regulating Medical Devices—Health 
Canada. We also did not audit goods imported as part of the CBSA’s 
trusted trader programs. 


1.13. We examined federal organizations’ systems and practices for 


* administering import requirements (permits, licences, certificates, 
or other authorizations) ; 


* targeting commercially imported consumer goods that pose a risk 
to the health and safety of Canadians; and 


* examining commercially imported consumer goods that pose a 
risk to the health and safety of Canadians. 


114 More details about the audit objectives, scope, approach, and 
criteria are in About the Audit at the end of this chapter. 


Observations and Recommendations 


115 We examined whether the Canada Border Services Agency 
(CBSA), the Canadian Food Inspection Agency (CFIA), Health 
Canada, Natural Resources Canada (NRCan), and Transport Canada 
administer import requirements for commercially imported products 
that pose a risk to the health and safety of Canadians by 


¢ documenting the administration of requirements; 


* following the roles, responsibilities, policies, and procedures for 
administering import requirements; and 


* monitoring the administration of requirements, as Treasury Board 
policies require. 


Most systems and practices for administering import requirements are well 
documented, but some need improvement 


1.16 Under the Canada Border Services Agency Act, the CBSA may enter 
into agreements or arrangements with federal organizations to provide 
services, such as administering import requirements, on their behalf. Such 
agreements document the import requirements, the products to which 
they apply, and the roles and responsibilities of the federal organizations 
involved in administering these requirements. Cooperation and 
coordination between the CBSA and the CFIA, NRCan, and Transport 
Canada are outlined in a series of bilateral memoranda of understanding 
(MOUs) that describe each party’s roles and responsibilities for collecting 
and exchanging information, targeting high-risk shipments, examining 
shipments, and monitoring compliance. 
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1.17 Cooperation and coordination between the CBSA and federal 
organizations are also described in a series of internal CBSA 
memoranda, called D Memoranda. These documents outline the 
legislation, policies, and procedures that the CBSA uses to administer 
specific customs programs, including the importation of certain goods 
that fall under the jurisdiction of other federal organizations. These 

D Memoranda also outline roles, responsibilities, and service standards. 
For the products included in our audit, current D Memoranda exist for 
those products under the responsibility of the CFIA, NRCan, and 
Transport Canada, and for some Health Canada products. 


1.18 Between 2000 and 2004, the CBSA stopped administering several 
Health Canada requirements at the border, including those for medical 
devices and pest control products and some for human-use drugs and 
controlled substances. These requirements were paper-based and 
required border services officers to process permits and other documents, 
and forward them to Health Canada. This practice was not consistent 
with the CBSA’s efforts to automate the importing process. The CBSA’s 
decision to stop administering Health Canada’s requirements meant that 
although legislation required importers to provide permits or import 
declaration forms with their shipments of some products, border services 
officers no longer reviewed the permits or forms. 


119. In 2003, Health Canada expressed concerns about the impact 
that the CBSA decision would have on the Department’s ability to 
fulfill its mandate to control imports of some products. The two 


‘ organizations therefore agreed to increase information exchange and 


targeting and to finalize an umbrella MOU that was already under 
development. The goal of the umbrella MOU was to clarify—for drugs, 
chemicals, pest control products, medical devices, and consumer 
products—the roles and responsibilities for collecting and exchanging 
information, targeting, examining shipments, and monitoring. The 
MOU was signed in November 2011, but no annexes have been agreed 
on. The annexes are an integral part of the MOU because they clarify 
roles, responsibilities, policies, and procedures of each participant for 
each program. Thus, while the MOU was an important first step, until 
the annexes are agreed on, the goal established in 2003 to clarify roles 
and responsibilities for control activities has not been met. 


1.20 Health Canada still expects that border services officers will 
refer shipments if they have any concerns about compliance with the 
Department’s import requirements. The CBSA, however, does not 
have the same expectation of its officers for all Health Canada 
products. As explained in the following section, we found that, in 
most cases where goods that did not meet their import requirements 
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were allowed to enter the country, they were products for which 
there is no agreement in place between Health Canada and the 
CBSA. A formal agreement would provide all border services officers 
with correct contact information and consistent instructions on 
procedures to follow, including when to refer a product and to which 
Health Canada program. 


1.21 Recommendation. To define and clarify roles, responsibilities, 
policies, and procedures for administering import requirements, the 
Canada Border Services Agency and Health Canada should prepare and 
finalize annexes for all programs identified in their 15 November 2011 
umbrella memorandum of understanding. The Canada Border Services 
Agency should then document or update instructions to border services 
officers for these programs and products. 


The Agency’s and the Department’s response. Agreed. The Agency 
and the Department will work together to complete annexes to the 
umbrella memorandum of understanding with related procedures that 
clearly define the roles and responsibilities of each organization for the 
border administration of the Department’s import requirements. One 
to two annexes would be completed by March 2013, with all 
completed by March 2014. 


Import requirements were administered correctly, with a few exceptions 


1.22 We assessed whether the federal organizations included in our 
audit follow the roles, responsibilities, policies, and procedures for 
administering import requirements. We examined a representative 
sample of 57 import transactions for regulated commercially imported 
products that were processed during May 2011. With respect to 
NRCan, we examined six randomly selected import transactions for 
fireworks, as there were too few fireworks transactions in May 2011 to 
include them in the representative sample. 


1.23 We found that the roles, responsibilities, policies, and procedures 
for administering import requirements were followed, except for two of 
the six fireworks transactions. When import requirements are not 
administered correctly, it does not always mean that the good is not 
compliant with safety regulations. For example, if an importer did not 
submit the required permit, it could mean either that the importer 
does not have a permit or that the importer has a permit but did not 
include it with the shipment. We therefore asked NRCan and the 
CBSA to review the transaction information for these two cases. 
They confirmed that, for one fireworks transaction, the importer did 
not have the necessary permit. 
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1.24 For five percent of transactions in our representative sample, 
administrative procedures were not strong enough for border officers to 
identify non-compliant goods. As a result, goods were allowed to enter 
the country, even though they did not meet their import requirements. 
These included a small quantity of an unregistered pest control product, 
and Class I medical devices (the lowest-risk class under the Medical 
Devices Regulations) when the importer did not have an appropriate 
establishment licence to demonstrate that it has the proper systems in 
place to safely manage the distribution of the device. These are goods 
for which there is currently no agreement between Health Canada and 
the CBSA to administer import requirements at the border. 


The Canada Border Services Agency’s framework to monitor administration of 
import requirements needs improvement 


1.25 The third element we examined, to assess whether the federal 
organizations included in our audit are administering import 
requirements at the border as intended, was their monitoring systems 
and practices. 


1.26 In 2009, the CBSA published an internal audit report on its 
administration of import requirements. The audit focused on a 
different set of controlled goods than those examined in our audit, but 
also found gaps in the administration of import requirements at the 
border. The audit therefore recommended that the Agency improve its 
monitoring of border controls. In response, the CBSA implemented a 
process monitoring framework. 


1.27. The framework is a checklist that requires supervisors to verify 
whether border services officers ensure that import requirements are 
met. The methodology provides different options to supervisors to 
conduct monitoring, depending on the size and complexity of the port 
of entry. However, for the results to be compared over time or from one 
location to another, supervisors need to report details on the 
methodology they used. We found, in most cases, they did not. We also 
found that there were few reports on results. Because the framework 
requires that only problems be reported, we could not determine 
whether the existence of few reports meant there were no problems to 
report, or the monitoring work had not been conducted. As a result of 
these design and implementation weaknesses, the framework does not 
provide the CBSA with the monitoring information it needs to detect 
changes in the effectiveness of border controls. 


1.28 The CFIA also has a process that is explicitly designed to monitor 
compliance with importing processes and requirements for fertilizers 
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Targeting imported 


consumer goods 


and supplements. The three other federal organizations rely mainly on 
their post-border monitoring processes, such as market surveillance, 
for assurance that their requirements are administered at the border as 
intended. We found that, despite the shared responsibility to control 
imports at the border, the CBSA and the other federal organizations 
included in our audit did not systematically share their monitoring 
information with each other. Sharing information about whether import 
requirements are being met, how often, and what types of problems 
need attention would help federal organizations identify whether 
changes to the effectiveness of this control have occurred, and where 
to direct resources and effort. 


1.29 Recommendation. The Canada Border Services Agency should 
ensure that the Process Monitoring Framework reporting includes 
enough information for the Agency to assess whether border controls 
are working as intended, share the results with relevant federal 
organizations, and act on the results. 


The Agency’s response. Agreed. The Agency will replace the Process 
Monitoring Framework with a new review structure. The new 
structure will streamline the Agency’s review process by aligning the 
review criteria and strategy with high-risk areas of business and audit 
priorities that affect port of entry operations nationally. The new 
structure will also include a log that records when issues and results are 
communicated to relevant organizations to ensure that issues are 
addressed. The Agency will develop and implement the new review 
structure by July 2012. 


1.30 When federal organizations identify high-risk commercially 
imported products that fall under their respective jurisdiction, they ask 
the Canada Border Services Agency (CBSA) to target and intercept 
these shipments. Targets are requested when, for example, an importer 
has previously brought in goods that were not properly declared or the 
federal organization has determined that a good is dangerous. 


1.31 We examined whether the CBSA, the Canadian Food Inspection 
Agency (CFIA), Health Canada, Natural Resources Canada 
(NRCan), and Transport Canada document systems and. practices for 
conducting targeting activities, follow targeting instructions as 
intended, and monitor the results. 


132 There are two types of targets: automated targets and border 
lookouts. An automated target is entered into ACROSS and is used to 
intercept a shipment based on targetable data, such as an importer’s 
business number. ACROSS automatically flags a shipment for 
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interception if the information the importer provides matches the 
characteristics defined in the target. Border services officers are 
expected to assess the target instructions and examine the shipment or 
refer it to the appropriate federal department, if required. 


1.33 However, shipments that the CBSA and the other federal 
organizations would like to target cannot always be identified in 
ACROSS. Some of the data items that ACROSS uses to target are not 
specific enough to identify all goods with import requirements. In these 
circumstances, the CBSA issues a border lookout, which is an 
instruction to border services officers to take specific action when they 
identify a shipment meeting established criteria. Border lookouts are 
communicated by email and all officers are expected to be aware of 
active lookouts. 


1.34 For the products included in our audit, for the period between 
November 2010 and June 2011, there were 250 automated targets and 
47 border lookouts, almost all of which were on products that Health 
Canada is responsible for. 


Instructions are being followed for automated targets, with a few exceptions 


1.35 We found that all five federal organizations document the way 
automated targets and border lookouts are to be conducted and follow 
the roles, responsibilities, policies, and procedures for creating targets. 
The CBSA then tests whether the automated target will intercept 
shipments as intended. 


1.36 To assess whether border services officers followed the target 
instructions, we selected a representative sample of 50 shipments from 
those intercepted as a result of an automated target between 

1 November 2010 and 30 June 2011. For 84 percent of transactions, 
officers followed the instructions that accompanied the target. In 

12 percent of transactions, all of which were targeted on behalf of 
Health Canada, border services officers detained and examined the 
shipments but did not refer them to the Department. Rather, they 
concluded that the goods did not match the target and released 
them. We asked Health Canada to verify whether its target 
instructions had been correctly followed in these cases. The 
Department stated that they had not been correctly followed and that 
it expected information about the shipments to be referred. However, 
its target instructions did not state that there was no room for officers’ 
discretion. These results reinforce the need for the two organizations 
to clarify their procedures for border controls. 


Chapter 1 15 


BORDER CONTROLS ON COMMERCIAL IMPORTS 


1.37 For the other four percent of transactions, also Health Canada 
targets, we found the goods were released without being detained, 
examined, or referred to the Department as instructed. The 
transactions included a natural health product that was suspected of 
containing lithium, which poses a health risk, and an importer whose 
shipments were all to be detained because of a history of non- 
compliance. For this four percent, CBSA officials could not explain 
why border services officers did not follow the target instructions. 


Information to monitor targets is inaccurate and incomplete 


1.38 The CBSA usually maintains automated targets and border 
lookouts for a limited period of time. It then asks the requesting federal 
organization whether the target should be removed, revised, or 
renewed. The requesting organization therefore needs a process to 
monitor the effectiveness of the target to guide its decision. This 
includes assurance that information about targeting activities and 
results is accurate and complete. . 


139 The CBSA’s National Risk Assessment Centre (NRAC) 
produces monthly monitoring reports about automated targets that 
are shared with the relevant federal organizations. However, the 
Agency does not analyze these reports, although its targeting policies 
and procedures require that it do so. Moreover, our review of the 
monthly monitoring reports found that some of the information was 
inaccurate or incomplete. For example, shipments that met target 
characteristics were recorded as intercepted, even though they had 
not been intercepted because the target had expired. As well, 
examination results included in the reports did not always show 
whether targeted shipments were compliant and therefore whether 
the targets were effective. 


1.40 ‘Transport Canada relies on the NRAC monthly reports to 
monitor the results of its targets and decide whether to renew, revise, 
or remove them. The CFIA, NRCan, and Health Canada supplement 
the monthly reports by directing their inspectors to watch for the 
targeted products as part of their market surveillance activities. 


1.41 Even though border lookouts are considered as high a risk as 
automated targets, we found that their results are not documented. 
Moreover, the federal organizations included in the audit do not 
have monitoring systems and practices in place to know whether 
these lookouts are implemented correctly or result in the 
interception of shipments. 
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142 These gaps in the information about targeting activities and 
results mean that, despite the time and resources each federal 
organization involved has invested, accurate and complete information 
is not available to effectively manage the targeting process. 


143 Recommendation. The Canada Border Services Agency 
should implement a quality assurance process for its automated target 
monitoring reports, and develop a means to monitor and report on the 
effectiveness of border lookouts. 


The Agency’s response. Agreed. The Agency will implement an 
effective quality assurance and monitoring process that will ensure 
accurate and complete reporting of results of other federal’ 
organizations’ targets and border lookouts on a timely basis. This will 
allow for timely sharing of information with relevant federal 
organizations about the effectiveness of the Agency’s actions on their 
behalf. Given that this process must reflect the business processes 
being introduced with the Agency’s implementation of a new targeting 


model, scheduled to begin on 1 April 2012, these measures will be 
completed by 30 June 2013. 


1.44 Examinations are conducted to confirm that shipments comply 
with requirements. There are two types of commercial examinations: 
document review and physical examination. During a document 
review, the Canada Border Services Agency (CBSA) sends the 
relevant federal organization the information that the commercial 
clients have submitted, including photocopies of labels or photographs 
the officers take of the product. The organization reviews the 
information and recommends that the CBSA refuse the shipment, 
release it, or examine it further. A physical examination ranges from a 
visual observation of the shipment to laboratory testing. 


1.45 Usually, border services officers conduct the examinations. As a 
result, the officers rely on instructions from the federal organizations to 
understand what to look for during the examinations. 


146 We examined whether the federal organizations included in 
our audit conduct document reviews and physical examinations at 


the border by 
* documenting the way examinations are to be conducted; 


* following the roles, responsibilities, policies, and procedures 
for examinations; and 


* monitoring examination activities and results. 


Report of the Auditor General of Canada—Spring 2012 


Chapter 1 17 


BORDER CONTROLS ON COMMERCIAL IMPORTS 


Procedures for examining high-risk products are well documented, 
with some exceptions 


147 The CBSA has a Customs Enforcement Manual, D Memoranda, 
and memoranda of understanding with other federal organizations that 
provide border services officers and superintendents with information 
about the policy, guidelines, and procedures for examining commercial 
shipments. These documents also include any service standards that 
have been agreed to between the CBSA and the other federal 
organizations. The memoranda generally instruct border services 
officers to contact the relevant federal organization for more 
information, and provide contact details. 


1.48 More detailed instructions on how to examine particular 
shipments are provided in target instructions that explain what action 
border services officers are to take once they have intercepted a 
shipment, or in the response federal organizations provide once officers 
refer a shipment to them. To assess whether the instructions that the 
federal organizations provided were the same as those that the CBSA 
entered in ACROSS, we selected a representative sample of 44 targets 
from those requested by Health Canada between November 2010 and 
June 2011. We also examined all 8 targets requested by the Canadian 
Food Inspection Agency (CFIA), Natural Resources Canada 
(NRCan), and Transport Canada for the same period. We found that 
most of the instructions were the same, but for five percent of the 

44 Health Canada targets, the instructions changed or excluded 
information. For one target, for example, the request from Health 
Canada was to detain two health products, but the CBSA’s target 
instructions listed only one of the products. 


Only Health Canada monitors referrals of shipments from the Canada Border 
Services Agency 


1.49 The CBSA, the CFIA, NRCan, and Transport Canada do not 
have systematic processes for recording when and why a border 
services officer refers a shipment, which shipment was referred, or 
any direction provided. As a result, the organizations cannot assess 
whether the CBSA followed their recommendations or whether they 
responded to referrals within agreed service standards. Therefore, we 
were unable to assess whether these federal organizations followed 
policies and procedures for referrals. 


1.50 In 2010, Health Canada established customs border centres in 
British Columbia and Ontario as pilot projects. The units are staffed by 
Health Canada officials who respond to and track CBSA referrals. 
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Prior to December 2011, the centres provided a single point of contact 
for all Health Canada product control programs, only in these two 
regions. In other regions, border services officers were still required to 
contact the Health Canada program responsible for each type of good 
directly, and Health Canada did not systematically document or track 
referrals. In December 2011, Health Canada expanded the single point 
of contact to all regions. 


1.51 Asa result of their more systematic approach to tracking 
referrals, the border centres can determine whether they meet the 
service standards that Health Canada set to respond to CBSA requests 
for assistance. 


1.52 Health Canada does not always record which shipment was 
referred by the CBSA, but the information the Department collected 
was sufficient to allow us to assess whether the CBSA followed the 
Department’s recommendations and refused shipments of health 
products. We reviewed a representative sample of 49 refusal 
recommendations. We found that the CBSA took the action 
recommended by Health Canada in 96 percent of transactions. 
However, in four percent, shipments of natural health products were 
released, even though Health Canada recommended the shipments be 
refused entry into Canada. 


1.53 Recommendation. The Canadian Food Inspection Agency, 
Health Canada, Natural Resources Canada, and Transport Canada 
should assess Canada Border Services Agency referrals systematically 
to determine whether responses are provided within agreed service 
standards and whether recommendations are followed. Each 
organization should use the results of this assessment to determine 
whether further action is required. 


The Canadian Food Inspection Agency’s response. Agreed. The 
Canadian Food Inspection Agency will conduct an assessment of its 
fertilizer referral process to the Canada Border Services Agency by 
Fall 2012 to determine if a formal monitoring program is required. 


Health Canada’s response. Agreed. Health Canada will develop 
service standards for each program as part of the development of 
program annexes with the Canada Border Services Agency (see 
response to recommendation 1.21). Health Canada also commits to 
monitoring i) compliance with agreed upon service standards, and 
ii) referrals from the Canada Border Services Agency in order to 
determine if the Agency is following Health Canada 
recommendations. 
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Natural Resources Canada’s response. Agreed. For the Explosives 
Program, a new recording system was developed in October 2011 and 
is currently being implemented by NRCan officers located in Ottawa 
for all referrals coming from border services officers. This is also being 
communicated by the end of Quarter 1, 2012, to all NRCan officers 
who may be contacted by border services officers. 


Transport Canada’s response. Agreed. Transport Canada has 
consulted with the Canada Border Services Agency and has agreed to 
review the appropriateness of current service standards with respect 
to referrals and make changes as required to the Memorandum of 
Understanding between the two organizations. The Department will 
also review the requirement to provide written confirmation regarding 
the status of vehicles within a one-hour time frame. This is expected to 


be completed by July 2012. 


Transport Canada and the Canada Border Services Agency have 
agreed that Transport Canada will create a tracking system to record 
referrals where Transport Canada, or its designated representative, has 
instructed the Canada Border Services Agency to deny entry of any 
goods referred. The system will record the Agency transaction number, 
the vehicle identification number(s), and the recommendation 


provided to the Agency. This system is expected to be in place by 
April 2012. 


Transport Canada will compare the referrals contained in the tracking 
system to those imported vehicles found to be inadmissible in any post- 
importation audit, and will forward these results to the Canada Border 
Services Agency quarterly for comment and resolution. 


Information is lacking on whether physical examinations produce intended results 


1.54 The CBSA has a Border Risk Management Plan, which aims to 
efficiently allocate the Agency’s resources to identify, interdict, and 
mitigate threats to border security. The plan states that accurately 
recorded results of physical examinations are necessary if the CBSA is to 
implement its risk management strategies properly. The CBSA therefore 
requires border services officers to record examination comments and 
codes in ACROSS. This information is also necessary to enable the 
CBSA to monitor whether border services officers have conducted the 
examinations that the other four federal organizations request. 


1.55 To assess whether officers record examination results correctly, 
we selected a representative sample of 50 transactions from May 2011 
in ACROSS. We also reviewed a non-representative random sample of 
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five import transactions for fireworks, as there were too few 
transactions to include them in the representative sample. 


1.56 We found that officers recorded examination results incorrectly 
or incompletely in 40 percent of transactions in the representative 
sample, and in all five fireworks transactions. For 20 percent of 
transactions and three fireworks cases, no examination comments were 
entered, contrary to CBSA policy. This means that there was no 
confirmation that the required examination was conducted to verify 
that the goods were as described and met import requirements. For 
example, in the case of an imported fertilizer targeted by the CFIA, 
there was a concern that the product contained phosphite, a substance 
considered a pest control product. Imported fertilizers containing this 
substance have different import requirements than other fertilizers. 
However, no examination results were entered. Therefore, there is no 
assurance that the border services officers checked to confirm the 
product did not contain phosphite. 


1.57 For the other 20 percent of transactions and the other two 
fireworks cases, an incorrect examination code was entered. The 
CBSA uses examination codes in reports to monitor the quality and 
effectiveness of examinations by, for example, analyzing the 
proportions of examinations that find non-compliant shipments. If the 
codes are inaccurate, the CBSA does not have correct information 
about the effectiveness of its examination activities. 


1.58 In addition to results generated from examination codes, the 
CBSA uses its Process Monitoring Framework to monitor whether 
border services officers documented the results of examinations and 
followed instructions from federal organizations. However, again, 
because of weaknesses in the framework, it does not provide the 
Agency with assurance that results are being recorded as required (see 
our recommendation in paragraph 1.29). 


1.59 We found a good practice concerning examination results for 
border blitzes. During our audit period, Health Canada conducted 
one border blitz on goods included in the scope of the audit at five 
ports of entry in Quebec. After the blitz, Health Canada and the 
CBSA shared the results of the exercise. In our opinion, sharing blitz 
results is a valuable way for each organization to understand how 
importers comply with import requirements and where to direct 
resources to stop non-compliant shipments in the future. 
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Single Window Initiative 


1.60 In 2005, the Canada Border Services Agency (CBSA) received 
$10 million to develop a Single Window Initiative. It is intended, in 
part, to improve control at the border by better identifying goods with 
import requirements, validating permits and other authorizations, and 
enabling the CBSA to exchange information with other federal 
organizations to better manage risk. The initiative is also intended to 
improve the administration of responsible federal organizations’ import 
requirements by allowing importers to submit information on their 
shipments once, electronically. Twelve federal organizations are 
participating in the Single Window Initiative. 


The Single Window Initiative to improve control at the border has achieved few 
of its goals 


1.61 We examined whether the Single Window Initiative was well 
managed according to best practices for information technology 
projects. We found that the CBSA had met few of its original project 
timelines and goals, and had not followed the project’s oversight 
process. 


1.62 At the time of the audit, the CBSA had begun sharing 
commercial customs data electronically with five federal organizations, 
including three of the federal organizations included in this audit, 
through a project called Pathfinder. Other Pathfinder links were under 
development. The project plan, schedule, and cost estimates were not 
completed until after project development had been under way for two 
years, and shortly before the initial projects were implemented. 


1.63 For a few programs, exchanging information through Pathfinder 
meets the goals of the participating federal organizations for the Single 
Window Initiative. However, for most of their programs, these 
organizations have concluded that they would benefit from achieving 
the Single Window Initiative’s other goals: better identification of 
goods, validation of permits and authorizations, and a single point for 
importers to submit information electronically. These organizations 
concluded in 2007 that they needed to develop a business case for 
additional funding. The CBSA initially intended to complete this 
business case in 2007, with a planned implementation date of 2010. 
However, the Agency and its partners did not complete necessary 
analyses in time. New deadlines for the business case (in 2008, 2009, 
and 2010) were also missed. At the time of the audit, a renewed effort 
was under way to complete a business case by the end of 2011 as part 
of the Perimeter Security and Economic Competitiveness Action Plan. 
In December 2011, the government announced that electronic decision 
processes would be implemented for Health Canada, the Canadian 
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Food Inspection Agency, Transport Canada, and Foreign Affairs 
and International Trade Canada through the Single Window Initiative 
by December 2013. 


Conclusion 


1.64 We found that, in most cases, the Canada Border Services 
Agency (CBSA), the Canadian Food Inspection Agency (CFIA), 
Health Canada, Natural Resources Canada (NRCan), and Transport 
Canada adequately control, at the border, selected commercially 
imported consumer goods that pose a risk to the health and safety of 
Canadians. Based on our review of transaction information from the 
CBSA’s databases, the controls to administer import requirements and 
to automatically target high-risk shipments are working as intended. 
That is, in almost all cases, roles, responsibilities, policies, and 
procedures were followed. 


1.65 We found, however, that improvements are needed in the 
management of border lookouts and examinations. Because their results 
are poorly documented, information was not available for us to 
determine whether these controls are working as intended. The CBSA 
does not keep a record of how many border lookouts result in shipments 
being stopped. As for examinations, although the CBSA requires that 
officers document examination results, this is not done consistently, and 
results are not always entered correctly. Consequently, it is not possible 
to determine whether border services officers carried out examinations 
in accordance with their instructions. Yet, border lookouts and 
examinations consume additional time and resources, and are reserved 
for shipments that are considered high risk. 


166 There are gaps in the monitoring practices for all three border 
controls—administration of import requirements, targeting of high- 
risk shipments, and examinations. This makes it difficult for federal 
organizations to know how well these controls are working and where 
resources and effort can be directed most effectively to manage risk. 
Specifically: 


¢ The CBSA’s Process Monitoring Framework is a checklist that 
requires supervisors to verify whether border services officers 
ensure that various border controls are working as intended. 
However, because of weaknesses in the framework, the results 
cannot be compared over time or from one location to another, 
and there were few reports on results. 
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¢ The CBSA’s target monitoring reports are inaccurate 


and incomplete. 


* The CBSA, the CFIA, NRCan, Transport Canada, and Health 
Canada in some regions do not have systematic processes for 
recording when and why a border services officer refers a 
shipment, which shipment was referred, or any direction provided. 
As a result, the organizations cannot assess whether the CBSA 
followed their recommendations or whether they responded to 
referrals within agreed service standards. 


1.67. The CBSA has formal arrangements with all four federal 
organizations in our audit. However, respective roles, responsibilities, 
policies, and procedures for implementing border controls are not yet 
documented for all Health Canada programs and products. In 2003, the 
CBSA and Health Canada committed to reaching an agreement that 
would clarify each party’s roles and responsibilities. A memorandum of 
understanding was signed in November 2011, but none of the annexes 
that clarify roles, responsibilities, policies, and procedures for each 
program has been agreed on. We found that, in most cases where goods 
were allowed to enter the country, even though they did not meet their 
import requirements, they were products for which there was no 
agreement in place between Health Canada and the CBSA at the 

time of the audit. A formal agreement would provide all border services 
officers with correct contact information and consistent instructions on 
procedures to follow, including when to refer a product and to which 
Health Canada program. 


1.68 Although under way since 2005, the Single Window Initiative 
to control at the border has met few of its original project timelines 
and goals. As well, the CBSA has not followed the project’s oversight 
process. In the meantime, the improvements that the CBSA says the 
initiative will make remain unrealized. 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these 
standards as the minimum requirement for our audits, we also draw upon the standards and practices of 
other disciplines. 


Objectives 


To determine whether the Canada Border Services Agency (CBSA), the Canadian Food Inspection 
Agency (CFIA), Health Canada, Natural Resources Canada (NRCan), and Transport Canada control, at 
the border, selected commercially imported products that pose a risk to the health and safety of Canadians. 


Scope and approach 


The audit scope focused on control at the border, to ensure key controls in the process of importing a 
shipment into Canada are operating as intended. These controls are the administration of import 
requirements, the targeting of high-risk shipments, and examinations. 


In this audit, we focused on consumer goods imported in significant volumes or those that might be 
difficult to assess for risk and compliance because of weak systems to flag problems. We therefore included 
in the scope of the audit the following federal organizations and the goods they regulate: 


- the CBSA; 
- the CFIA, for fertilizers and supplements; 


« Health Canada, for pharmaceuticals, non-prescription medicines, natural health products, medical 
devices, regulated and non-regulated consumer products, and pest control products; 


« NRCan, for consumer fireworks; and 


¢ Transport Canada, for vehicles and tires. 


Commercially imported consumer goods refers to those that, with their components, parts, accessories, 
and packaging, are imported into Canada through the CBSA’s commercial stream and that may reasonably 
be expected to be obtained by an individual and used for non-commercial purposes, including for 
domestic, recreational, and sports activities. 


Goods imported through CBSA’s trusted trader programs were excluded from the scope of the audit. We 
also did not examine the other product control processes that the audited organizations may use to ensure 
the safety of imported goods, such as the licensing and inspection of manufacturing or distribution 
facilities, or market surveillance activities. We did not examine the responsible federal entities’ processes 
for issuing licences, permits, or other authorizations needed to import goods, nor their processes for 


identifying high-risk goods to be targeted by the CBSA. 


We interviewed officials, reviewed documents, and conducted process and system walk-throughs at 
the national headquarters of the five federal organizations included in the scope of the audit. We also 
interviewed officials and reviewed documents in regional offices of the CBSA, the CFIA, and 
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Health Canada in British Columbia, Ontario, and Quebec—the regions through which the majority of 
imported goods enter Canada. We did not visit regional offices of Transport Canada and NRCan, because, 
for the programs included in our audit, the departments conduct most of their import control activities 


centrally. 
In addition, we conducted file reviews of four representative samples of import transactions. 


+ To assess whether import requirements were administered as intended, we randomly selected 
57 import transactions for goods regulated by the CFIA, Health Canada, and Transport Canada, from 
a total of 29,951 import transactions for these goods processed in May 2011. 


- To assess whether automated target instructions were followed as required, we randomly selected 
50 import transactions out of a population of 3,635 import transactions for goods regulated by the 
CFIA, Health Canada, NRCan, and Transport Canada: These import transactions contained goods 
that had been intercepted by an automated target between November 2010 and June 2011. 


* To assess whether information about examinations was correctly recorded, we randomly selected 
50 import transactions out of a population of 2,323 import transactions for goods regulated by the 
CFIA, Health Canada, and Transport Canada. These import transactions contained goods that had 
been referred for examination in May 2011. 


- To assess whether goods recommended for refusal were refused entry, we randomly selected 49 import 


transactions out of a population of 1,201 transactions that had been recommended for refusal by 
Health Canada between November 2010 and June 2011. 


Each of the random samples is considered accurate within 10 percent, 9 times out of 10. 


We randomly selected six additional fireworks transactions to examine the effectiveness of the controls for 
the administration of import requirements, and five for examinations. This information could not be 
obtained from the representative samples, because there were too few NRCan transactions in each of the 
sampled populations. 


‘To assess whether target instructions were properly entered into ACROSS, we conducted a file review of 
44 automated targets from a population of 242 targets requested by Health Canada between 

November 2010 and June 2011. We also examined all of the 8 automated targets requested by the CFIA, 
NRCan, and Transport Canada for the same period. 
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Criteria 


consumer goods that pose a risk to the health and safety of 
Canadians by 


documenting the administration of relevant federal 
organization requirements; 


following the roles, responsibilities, policies, and procedures 
for administering pertinent federal organization requirements; 
and 


monitoring the administration of relevant federal organization 
requirements. 


The federal organizations administer relevant requirements for 
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SES Border ee Agency Act 
Customs Act 

Controlled Drugs and Substances Act 
Explosives Act 

Fertilizers Act 

Food and Drugs Act and Regulations 
Hazardous Products Act 

Motor Vehicle Safety Act 

Pest Control Products Act 


Policy on Management, Resources and Results Structures, 
Treasury Board, 2010 


Policy on Information Management, Treasury Board 


The Canada Border Services Agency manages the Single Window 
Initiative consistent with IT project management best practices by 


establishing a governance and accountability structure, 


managing the project based on risk management principles, 


identifying key stakeholders and defining their needs, 


aligning the project with the business strategy of the Agency, 
and 


assessing its capacity to deliver on the project. 


Policy on the Management of Projects, Treasury Board 
Policy on Information Management, Treasury Board 


Directive on Management of Information Technology, 
Treasury Board 


Standard for Organizational Project Management Capacity, 
Treasury Board 


Control Objectives for Information and Related Technology, 
IT Governance Institute 


The federal organizations target and intercept consumer goods 
that pose a risk to the health and safety of Canadians by 


¢ documenting how targeting activities are to be conducted; 


* following the roles, responsibilities, policies, and procedures 
for targeting; and 


* monitoring targeting activities and results. 


Canada Border Services Agency Act 
Customs Act 
Framework for the Management of Risk, Treasury Board 


Policy on Management, Resources and Results Structures, 
Treasury Board, 2010 


Policy on Information Management, Treasury Board 
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The federal organizations examine, at the border or through - ¢ Canada Border Services Agency Act 
follow-up examinations, consumer goods that pose a risk to the 
health and safety of Canadians by 


Customs Act 


e 


: d Substances Act 
* documenting how examinations are to be conducted; Gonuolied 21s 2) 


* following the roles, responsibilities, policies, and procedures Explosives Act 


for examinations; and 


Fertilizers Act 


¢ monitoring examination activities and results. 


Food and Drugs Act 


Hazardous Products Act 


Motor Vehicle Safety Act 


Pest Contro! Products Act 


Framework for the Management of Risk, Treasury Board 


Policy on Management, Resources and Results Structures, 
Treasury Board, 2010 


Policy on Information Management, Treasury Board 


Management reviewed and accepted the suitability of the criteria used in the audit. 


Period covered by the audit 


The audit focused on the Canadian import control regime during the period November 2010 to 
September 2011. However, some documents reviewed go back to 2003. Audit work for this chapter was 
substantially completed on 30 September 2011. 


Audit team 


Assistant Auditor General: Wendy Loschiuk 
Principal: Nicholas Swales 
Director: Joanne Butler 


Jared Albu 

Bernard Battistin 
Chantal Descarries 
Eamonn Gaffney 
Tammi Martel 
Anthony Stock 
Hugues-Albert Sully 
Randolph Young 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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BORDER CONTROLS ON COMMERCIAL IMPORTS 


The following is a list of recommendations found in Chapter 1. The number in front of the 
recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses 
indicate the paragraphs where the topic is discussed. 


Administering import requirements 


1.21 To define and clarify roles, 
responsibilities, policies, and procedures 
for administering import requirements, 
the Canada Border Services Agency 
and Health Canada should prepare and 
finalize annexes for all programs 
identified in their 15 November 2011 
umbrella memorandum of 
understanding. The Canada Border 
Services Agency should then document 
or update instructions to border services 
officers for these programs and 
products. (1.16—1.20) 


1.29. The Canada Border Services 
Agency should ensure that the Process 
Monitoring Framework reporting 
includes enough information for the 
Agency to assess whether border 
controls are working as intended, share 
the results with relevant federal 
organizations, and act on the results. 


(1.25-1.28) 


The Agency’s and the Department’s response. Agreed. 

The Agency and the Department will work together to complete 
annexes to the umbrella memorandum of understanding with 
related procedures that clearly define the roles and responsibilities 
of each organization for the border administration of the 
Department’s import requirements. One to two annexes would be 


completed by March 2013, with all completed by March 2014. 


The Agency’s response. Agreed. The Agency will replace 

the Process Monitoring Framework with a new review structure. 
The new structure will streamline the Agency’s review process 
by aligning the review criteria and strategy with high-risk areas 
of business and audit priorities that affect port of entry 
operations nationally. The new structure will also include a log 
that records when issues and results are communicated to 
relevant organizations to ensure that issues are addressed. The 
Agency will develop and implement the new review structure by 


July 2012. 
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Targeting imported consumer goods 


1.43 The Canada Border Services 
Agency should implement a quality 
assurance process for its automated 
target monitoring reports, and develop 


a means to monitor and report-on the ~ 


effectiveness of border lookouts. 
(1.38-1.42) 


The Agency’s response. Agreed. The Agency will implement 
an effective quality assurance and monitoring process that will 
ensure accurate and complete reporting of results of other 
federal organizations’ targets and border lookouts on a timely 
basis. This will allow for timely sharing of information with 
relevant federal organizations about the effectiveness of the 
Agency’s actions on their behalf. Given that this process must 
reflect the business processes being introduced with the 
Agency’s implementation of a new targeting model, scheduled 
to begin on 1 April 2012, these measures will be completed by 
30 June 2013. 


Examining imported consumer goods 


1.53 The Canadian Food Inspection 
Agency, Health Canada, Natural 
Resources Canada, and Transport 
Canada should assess Canada Border 
Services Agency referrals systematically 
to determine whether responses are 
provided within agreed service standards 
and whether recommendations are 
followed. Each organization should use 
the results of this assessment to 
determine whether further action is 
required. (1.49-1.52) 


The Canadian Food Inspection Agency’s response. Agreed. 
The Canadian Food Inspection Agency will conduct an 
assessment of its fertilizer referral process to the Canada Border 
Services Agency by Fall 2012 to determine if a formal 
monitoring program is required. 


Health Canada’s response. Agreed. Health Canada will develop 
service standards for each program as part of the development of 
program annexes with the Canada Border Services Agency (see 
response to recommendation 1.21). Health Canada also 
commits to monitoring i) compliance with agreed upon service 
standards, and ii) referrals from the Canada Border Services 
Agency in order to determine if the Agency is following Health 
Canada recommendations. 


Natural Resources Canada’s response. Agreed. For the 
Explosives Program, a new recording system was developed in 
October 2011 and is currently being implemented by NRCan 
officers located in Ottawa for all referrals coming from border 
services officers. This is also being communicated by the end of 
Quarter 1, 2012, to all NRCan officers who may be contacted by 
border services officers. 


Transport Canada’s response. Agreed. Transport Canada has 
consulted with the Canada Border Services Agency and has 
agreed to review the appropriateness: of current service standards 
with respect to referrals and make changes as required to the 
Memorandum of Understanding between the two organizations. 
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The Department will also review the requirement to provide 
written confirmation regarding the status of vehicles within a 
one-hour time frame. This is expected to be completed by 


July 2012. 


Transport Canada and the Canada Border Services Agency have 
agreed that Transport Canada will create a tracking system to 
record referrals where Transport Canada, or its designated 
representative, has instructed the Canada Border Services 
Agency to deny entry of any goods referred. The system will 
record the Agency transaction number, the vehicle identification 
number(s), and the recommendation provided to the Agency. 
This system is expected to be in place by April 2012. 


Transport Canada will compare the referrals contained in the 
tracking system to those imported vehicles found to be 
inadmissible in any post-importation audit, and will forward 
these results to the Canada Border Services Agency quarterly for 
comment and resolution. 
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CHAPTER 2 
Replacing Canada’s Fighter Jets 


Performance audit reports 


This report presents the results of a performance audit conducted by the Office of 
the Auditor General of Canada under the authority of the Auditor General Act. 


A performance audit is an independent, objective, and systematic assessment 

of how well government is managing its activities, responsibilities, and resources. 
Audit topics are selected based on their significance. While the Office may 
comment on policy implementation in a performance audit, it does not comment 
on the merits of a policy. 


Performance audits are planned, performed, and reported in accordance with 
professional auditing standards and Office policies. They are conducted by 
qualified auditors who 


¢ establish audit objectives and criteria for the assessment of performance; 
¢ gather the evidence necessary to assess performance against the criteria; 
* report both positive and negative findings; 

* conclude against the established audit objectives; and 


* make recommendations for improvement when there are significant 
differences between criteria and assessed performance. 


Performance audits contribute to a public service that is ethical and effective 
and a government that is accountable to Parliament and Canadians. 


Table of Contents 


Main Points 


Introduction 


The United States-led Joint Strike Fighter Program 
Canada’s fighter jet acquisition process 
Focus of the audit 


Observations 


Participating in the Joint Strike Fighter Program 


Early efforts to promote industrial participation were successful 
Projections of industrial benefits fluctuated and uncertainties were not communicated 
Preliminary analysis was conducted to consider various fighter jet options 


Risks associated with participating in the Joint Strike Fighter Program were communicated 
to decision makers 


There was a lack of consultation and communication on the procurement regime 


Procuring new fighter jets 


Key decision-making steps and documents were of little consequence 
Risks and mitigating strategy did not reflect cost increases and delays 
Full life-cycle costs were not presented and were likely underestimated 


Recommendation 
Conclusion 
About the Audit 


Appendix 


Recommendation 
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What we examined 
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Main Points 


Canada currently operates a fleet of CF-18 Hornet fighter jets purchased 
in the 1980s with an original life expectancy until 2003, since extended 
to between 2017 and 2020 after a modernization program in early 2000. 
National Defence has determined that a suitable replacement for the 
CF- 18 is required, or it will lose its ability to carry out domestic and 
international missions mandated by the Government of Canada. 


National Defence has been a partner in the Joint Strike Fighter (JSF) 
Program since 1997. Led by the United States, and with eight other 
country partners, the Program is undertaking concurrent design, 
development, and manufacturing of the F-35 Lightning II aircraft. It will 
eventually include a regime for long-term collaborative sustainment. 
Canada’s participation has been formalized by signing international 
memoranda of understanding—in 1997, 2002, and 2006—for each of 
the three major phases of the JSF Program. As of September 2011, the 
government had disbursed about CAN$335 million toward participation 
in the JSF Program and related support to Canadian industry. The 
government has committed a total of US$710 million to the Program. 


In May 2008, through the Canada First Defence Strategy, the federal 
government announced its intent to replace the CF-18 fleet with 

65 “next generation” fighter aircraft. Then, in July 2010, the 
government announced its decision to buy the F-35 Lightning II, 
without following a competitive process, as the CF-18 replacement. 


We examined whether National Defence, Industry Canada, and Public 
Works and Government Services Canada exercised due diligence in 
managing Canada’s participation in the JSF Program and in managing 
the federal decision-making process to acquire the F-35 as a 
replacement for the CF-18. 


Our conclusions relate only to the management practices and actions 
of public servants. We did not audit private sector contractors and, 
consequently, our conclusions do not pertain to the contractor’s 
practices or to their performance. We did not audit the merits of 

the F-35 aircraft. 
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Why it’s important 


What we found 


Audit work for this chapter was substantially completed on 
30 September.2011. Further details on the conduct of the audit are 
in About the Audit at the end of this chapter. 


Buying major defence equipment is subject to decision-making and 


project management processes whose aim is to ensure that decisions 
are well founded, projects are managed effectively, and goods and 
services are acquired in a way that enhances supplier access, 


competition, and fairness. National Defence, Industry Canada, 
Public Works and Government Services Canada (PWGSC), and 


central agencies are involved. 


Buying and maintaining the F-35, or any other fighter jet, will require a 


significant long-term financial commitment. The F-35 is still being 


developed and tested, and projections of purchase price and 


sustainment costs are still being refined. Decisions taken to date as well 


as those yet to come will have impacts for the next 40 years. 


National Defence took the appropriate steps in managing Canada’s 
participation in the Joint Strike Fighter (JSF) Program to develop the 
F-35. National Defence engaged Industry Canada early, and together 
they managed industrial participation well. Early efforts to secure 
contract opportunities for Canadian companies were successful. 


There were significant weaknesses in the decision-making process 
used by National Defence in acquiring the F-35 to replace the 
CF-18. By the end of 2006, the Department was actively involved in 
developing the F-35, and a number of activities had put in motion its 
eventual procurement. In the lead-up to the government’s 2010 
announcement, required documents were prepared and key steps 
were taken out of sequence. Key decisions were made without 
required approvals or supporting documentation. 


PWGSC did not fully carry out its role as the government's 
procurement authority. Although it was not engaged by National 
Defence until late in the decision-making process, PWGSC endorsed 
the key decision to sole source the acquisition of the F-35 in the 
absence of required documentation and completed analyses. By that 
time, practically speaking, Canada was too involved with the aircraft 
and the JSF Program to run a fair competition. 


National Defence did not provide complete information in a timely 
manner. For example, briefing materials prepared for decision makers 
did not explain the basis for and limitations of projections of 
industrial benefits to Canadian companies, and the risks of relying on 
the projections for decision making. In addition, briefing materials 
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did not inform senior decision makers, central agencies, and the 
Minister of the problems and associated risks of relying on the F-35 
to replace the CF-18. Nor did National Defence provide complete 
cost information to parliamentarians. 


National Defence likely underestimated the full life-cycle costs of 

the F-35. The budgets for the F-35 acquisition (CAN$9 billion) and 
sustainment (CAN$16 billion) were initially established in 2008 
without the aid of complete cost and other information. Some of that 
information will not be available until years from now. If the budgets 
prove insufficient to cover total costs, the Department will have to 
find ways to cover additional costs that may be incurred. Alternatively, 
it may have to seek additional funds from the government or use funds 
from other parts of its capital or operating budgets. 


The departments have responded. National Defence agrees with our 
recommendation. Its response follows the recommendation. 


National Defence, Industry Canada, and Public Works and 
Government Services Canada have accepted the facts presented in the 
chapter. Both National Defence and Public Works and Government 
Services Canada disagree with the conclusions set out in paragraphs 


2.80 and 2.81. 
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CF-18 Hornet 


Photo: Department of National Defence 


F-35 Lightning II 


Photo: Lockheed Martin Aeronautics 


Sustainment—Aircraft maintenance and repair, 
provision of spare parts, technological upgrades, 
training, and other operating activities. 
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Introduction 


2.1. Canada purchased its current fleet of CF-18 Hornet fighter jets in 
the 1980s, with an original estimated life expectancy to 2003. A major 
modernization program carried out in the early 2000s extended the life 
of the aircraft until between 2017 and 2020. According to National 
Defence, when the CF-18 is retired at that time, unless there is a 
suitable replacement, Canada will lose its fighter jet capability and 
consequently its ability to carry out domestic and international missions 
mandated under the 2008 Canada First Defence Strategy. 


2.2 In July 2010, the Government of Canada announced its decision 
to buy 65 F-35 Lightning II jets to replace Canada’s CF-18 fleet. The 
announcement was the culmination of nearly 13 years of Canada’s 
participation in the United States-led Joint Strike Fighter Program. 
Buying and maintaining the F-35, or any other fighter jet, will require a 
significant long-term financial commitment. It will have far-reaching 
economic and operational impacts on Canadians and the Canadian 
Forces. Decisions taken to date as well as those yet to come will have 
impacts for the next 40 years. 


The United States-led Joint Strike Fighter Program 


2.3. The Joint Strike Fighter (JSF) Program started in the late 1990s. 
It is the United States Department of Defense’s largest-ever 
development and procurement program, aiming to produce an 
affordable, multi-role aircraft able to perform in combat operations. 
The JSF Program is unique. Led by the United States, and with 

eight international partners (Australia, Canada, Denmark, Italy, 
Netherlands, Norway, Turkey, and United Kingdom), the JSF Program 
is undertaking concurrent design, development, and manufacturing of 
the F-35. It will eventually include a regime for long-term sustainment. 


2.4 Supporters of the F-35 expect it to be the most technologically 
advanced strike fighter jet in the world. Three variants of the aircraft 
will be produced: 


* conventional takeoff and landing, 
° carrier, and 


¢ short takeoff/vertical landing. 


2.5 These variants are intended to replace existing fighter jets 
operated by the United States Air Force, Navy, and Marine Corps and 
other militaries around the world. Together, the nine partners are 
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anticipated to buy about 3,100 aircraft, with the United States 
acquiring 2,443. Canada plans to purchase 65 jets, of the conventional 
takeoff and landing variant. 


26 The JSF Program is divided into three major phases. At the 
beginning of each phase, country partners formalized their participation 
and financial contribution by signing a government-to-government 
memorandum of understanding (MOU). The following are the 

three phases: 


* concept demonstration (1997 to 2001); 
* system development and demonstration (2001 to 2018); and 


* production, sustainment, and follow-on development (2006 
to 2051). 


21. The JSF Program was conceived to be a new model for 
development and acquisition of military equipment. It has the 
following features, some of which are unique: 


¢ Industrial participation. Companies from partner countries are 
eligible to participate in the manufacturing and sustainment over 
the lifetime of all jets produced. There is no guaranteed work for 
companies from partner countries. 


¢ Acquisition methods. There are two ways to buy the F-35: 
through the third-phase 2006 MOU as a partner or through 
foreign military sales as a non-partner (at a higher price). In both 
cases, the purchase occurs through the US government. There is 
no contract directly between partner countries and the 
manufacturers. The 2006 MOU recognizes that procurement 
is subject to partners’ national laws and policies. 


* Purchase costs. The purchase price paid for the basic aircraft in 
any given year will be the same for all partners, including the US 
military. The purchase price does not include the substantial 
development costs incurred by the United States government. 
Partners will have to pay the cost of designing and manufacturing 
any requested modifications to their basic aircraft. 


* Input and access. Partner countries have input on the design of 
the aircraft and the JSF Program through participation on 
decision-making and technical committees. Canada’s contribution 
gives National Defence access to the newest technologies and 
some highly technical information granted only to partners. 
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First operational aircraft—A production jet 
that does not contain specialized equipment 
required for operational testing. This is in 
contrast to a development jet that is equipped 
with specialized equipment to monitor the jet’s 
performance during testing. 


Unit recurring flyaway cost—The cost of an 
aircraft's airframe, vehicle and mission 
systems, and propulsion. 
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¢ Sustainment. For maintenance and repair, provision of spare 
parts, technological upgrades, training, and other operating 
activities, partner countries and industry will be part of a global 
network. The sustainment strategy is still being developed. 


2.8 The Joint Strike Fighter Program Office plays a critical role in 
managing the JSF Program. It provides a forum for decision making, 
information sharing for all partners, information technology support, 
and management of the project on behalf of participants. National 
Defence has representatives on the JSF Program senior leadership, 
financial, and technical committees. Notwithstanding its international 
nature, the JSF Program is US-led and is subject to US law, rules, and 
Congressional oversight. 


2.9 The JSF Program has experienced cost increases, schedule 
delays, and technological problems. These issues are not uncommon 
for a major development program. As of April 2011, the US 
Government Accountability Office estimated total development 
funding to be US$56.4 billion to complete the project by 2018, 
which is a 64 percent increase in estimated costs since the system 
development and demonstration phase started. 


2.10 The number of F-35 jets being produced and delivered is 
expected to increase over time. Initially, small numbers are being 
produced while the aircraft design is being tested and modified, 
and operational capabilities are added. These are referred to as first 
operational aircraft. Once the design is stable, and it has been 
demonstrated that the aircraft is capable of being deployed into a 
hostile situation, it will be subject to further operational testing and 
evaluation by US authorities. Once approved, the manufacturing 
process will enter its full-rate production. At the time of this audit, 
the aircraft was still in development and many technical challenges 
were yet to be overcome. 


2.11 The JSF program has been subject to several major technical, 
cost, and manufacturing reviews and has been significantly adjusted 
three times. Exhibit 2.1 shows changes in delivery schedule estimates 
of the F-35 conventional takeoff and landing variant, from the start of 
the system development and demonstration phase in October 2001 to 
April 2010. Exhibit 2.2 shows the increases in the estimated unit 
recurring flyaway cost for the conventional takeoff and landing 
variant over the full production period until 2035. 


Chapter 2 7 


REPLACING CANADA'S FIGHTER JETS 


8 Chapter 2 


2.12 In July 2010, US authorities began another comprehensive 
review of the JSF Program. This review is expected to affect the unit 
recurring flyaway cost, sustainment costs, as well as the development, 
production, and delivery schedule. At the time of this audit, partner 
countries were awaiting the outcome of this review. Exhibits 2.1 

and 2.2 do not reflect the outcome of this review. 


Exhibit 2.1 There have been three changes from 2001 to 2010 in estimated delivery dates for the F-35 
(conventional takeoff and landing variant) 


First 
operational 


: 2008 2010 
aircraft 
delivery 
Full-rate 9012 


production 


This exhibit does not reflect the outcome of the United States (US) review 
of the Joint Strike Fighter Program. 


Source: US Government Accountability Office (April 2011) 


Exhibit 2.2 Estimates for the full production period of unit recurring flyaway costs increased 
from 2001 to 2009 


US$ millions 
100 


90 
80 


70 


Oct-01 Dec-02 Dec-03 Dec-04 Dec-05  Dec-06 Dec-07 Dec-08  Dec-09  Dec-10* 
Years 


* Figure is not available—awaiting outcome of the United States (US) review of the Joint Strike Fighter Program. 


Amounts are in US millions of “then year” dollars, the equivalent of Canadian budget-year dollars. 
Unaudited figures 


Source: Joint Strike Fighter Program Office 
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Canada’s fighter jet acquisition process 


2.13 The federal process to procure major defence equipment is 
governed by legislation, policy, and departmental guidance. Together, 
these specify the respective roles of federal departments and agencies, 
the key steps to be followed, the types of information and mandatory 
documents required, and the approvals that must be obtained as a 
procurement proceeds. 


2.14 In May 2008, the federal government released the Canada First 
Defence Strategy, which signalled its intent to replace the CF-18 fleet 
with 65 “next generation” fighter aircraft. In July 2010, the 
government announced its decision to buy 65 F-35 aircraft at a cost of 
CAN$9 billion, without following a competitive process. National 
Defence expects the first aircraft to arrive in Canada in 2019. 


Focus of the audit 


2.15 This audit examined whether the three main departments 
involved in acquiring military equipment—National Defence, 
Industry Canada, and Public Works and Government Services Canada 
(the government’s contracting authority)—applied due diligence in 
managing Canada’s participation in the Joint Strike Fighter Program 
and managing the federal decision-making process to replace the 
CF-18 fighter jets. We define “due diligence” to mean that the 
departments have, in support of key decisions and related 
management activities, 


* performed and documented analyses (of benefits and risks, 
operational requirements, options, and costs); 


¢ clarified rules, roles, and responsibilities; 
¢ consulted with other entities; and 


* obtained approvals and provided oversight. 


2.16 Our criteria reflect basic principles of good management 
practices and due diligence, and are based on relevant Treasury Board 
policies that support decision making and sound stewardship, and that 
contribute to transparency, accountability, and value for money. 


2.17 More details about the audit objectives, scope, approach, 
and criteria are in About the Audit at the end of this chapter. 
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Participating in the Joint Strike 
Fighter Program 


Budget-year dollars—Costs in budget-year 
dollars reflect the purchasing power of the dollar 
in the year the cost is incurred. This is in 
contrast to costs in constant-year dollars, which 
reflect prior year, current, and future costs at the 
level of prices of a defined base year. 
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Observations 


218 Our recommendation is in paragraph 2.77. 


2.19 To date, Canada has been involved in the Joint Strike Fighter 
(JSF) Program for almost 15 years. Officials from National Defence 
have contributed to all three phases, including participating in its 
senior decision-making and technical committees. In addition, 
Industry Canada, together with National Defence, made efforts to 
ensure Canadian companies had (and continue to have) opportunities 
to bid on work in connection with the aircraft development as well as 
eventual production and sustainment. There is no single set of federal 
policies or rules that govern participation in an international initiative 
such as the JSF Program. Exhibit 2.3 provides a timeline of key events 
in the JSF Program and Canada’s F-35 procurement process. 


2.20 National Defence signed the memorandum of understanding 
(MOU) for the first phase of the JSF Program, concept demonstration, 
in December 1997. Its US$10.6 million (budget-year dollars) 
contribution was made within the expenditure authority of the 
Minister of National Defence and did not require approval by the 
Treasury Board. Faced with the eventual need to replace the CF-18 
fighter jets, National Defence felt its participation provided the chance 
to work with allies in developing a new fighter jet as well as 
opportunities for Canadian companies to be part of the design and 
eventual production of an estimated 3,100 aircraft (if Canada decided 
to purchase the aircraft). 


2.21 In October 2001, the United States Department of Defense 
selected Lockheed Martin as the winner of a design competition held 
between two major industry consortia. National Defence signed the 
MOU for the second phase, system development and demonstration, 
in February 2002. In total, Canada committed to contributing 
US$150 million (budget-year dollars) for this phase; Treasury Board 
approved US$100 million in December 2001, which National Defence 
provided directly to the JSF Program. A further US$50 million was 
distributed to Canadian industry through pre-existing Industry Canada 
programs that were mandated to provide funding support to strategic 
research and development projects. According to National Defence, 
securing industrial benefits for Canadian companies continued to bea 
driving motivation for participation. 
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2.22 In 2006, the JSF Program entered its third phase: production, 
sustainment, and follow-on development. This marked an important 
transition to a regime for producing, procuring, and maintaining the 
F-35. In our view, 2006 also represented the most critical period 
concerning Canada’s participation in the JSF Program and future 
acquisition of the F-35. 


2.23 In November 2006, Industry Canada signed industrial 
participation memoranda of understanding with the JSF Program’s 
prime contractors, the US-based companies that will manufacture the 
jets and engines: Lockheed Martin, Pratt @ Whitney, and GE Rolls- 
Royce. These MOUs formalized a framework for offering opportunities 
to Canadian companies to be part of the supply chain to produce the 
jets and engines as subcontractors. Most importantly, National Defence 
signed the third-phase MOU in December 2006. This signing 
represented a significant financial commitment to the Program—up to 
US$551 million over 40 years, provided Canada remains a partner— 
to be used for, among other things, manufacturing test and tooling 
equipment. It also represented Canada’s acceptance of the procurement 
regime for the F-35. The Treasury Board approved a portion of Canada’s 
contribution to the JSF Program (CAN$182 million to 2013) in 
November 2006. It also approved CAN$10 million to support National 
Defence’s internal Program Office costs. As of September 2011, 

the government had disbursed a total of CAN$335 million toward 
participation in the JSF Program and support to Canadian industry. 
National Defence plans to seek expenditure authority for the remainder 
of the contribution at a later date. 


2.24. According to National Defence, ongoing partnership in the JSF 
Program, and the eventual purchase of the F-35 through the provisions 
of the 2006 MOU, offered significant benefits. In addition to industrial 
benefits, these included 
* unprecedented access to data about next generation fighter aircraft, 
* cost avoidance (not having to pay for research and development 
costs or the fees associated with foreign military sales), 
* savings on long-term sustainment costs through a collaborative 
approach with other partners, 
¢ military “interoperability” (ability to work with allies and their 
equipment), and 


* potential royalties on sales of the F-35 to non-partner countries. 
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Exhibit 2.3 Timeline of key events for the Joint Strike Fighter (JSF) Program and the F-35 procurement 


Canada—JSF Program and F-35 procurement 


National Defence signed JSF Program’s first-phase 


memorandum of understanding (MOU) 


National Defence began CF-18 modernization 


National Defence, Industry Canada, and others signed an 
interdepartmental MOU defining roles and responsibilities 


Treasury Board approved CAN$171 million for second JSF 
Program phase 


National Defence signed JSF Program’s second-phase MOU 


National Defence assigned Canadian representative to JSF 
Program office 


National Defence completed a preliminary options analysis of five 
candidate aircraft 


Industry Canada signed industrial participation MOUs with US 
prime contractors 


Treasury Board approved CAN$192 million (to 2013) for 
Canada’s participation in JSF Program’s third phase 


National Defence signed JSF Program's third-phase MOU 


Government released Canada First Defence Strategy 


National Defence started work to replace CF-18s, including 
identifying 14 high-level mandatory capabilities 


National Defence completed options analysis of three contender 
aircraft against high-level capabilities 


National Defence sought government's decision to purchase F-35, 
but process was put on hold 


Treasury Board approved five-year investment plan for National 
Defence, which included National Defence’s plan to buy next 
generation fighter aircraft 


12 


United States (US) and international—JSF Program 


r 1997 ) First phase (concept demonstration) began 


US selected Lockheed Martin as prime 


contractor to build the F-35 


First phase ended 


Second phase (system development and 
demonstration) began 


2002 4 


Technical, costing, and manufacturing review 
led to first JSF Program adjustment 


Third phase (production, sustainment, and 
follow-on development) began 


2007 4 Second JSF Program adjustment 
2008 Independent costing review of JSF Program 


Independent costing and manufacturing reviews 
of JSF Program 


Chapter 2 
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Exhibit 2.3 Timeline of key events for the Joint Strike Fighter (JSF) Program and the F-35 procurement (continued) 


Canada—JSF Program and F-35 procurement United States (US) and international—JSF Program 


National Defence officially began Next Generation Fighter : : 
Capability project to replace CF-18s $2010 ie peeeralneciustiieit 


US began comprehensive independent technical 
review of detailed plans that will result in 
additional program restructuring. 


Officials in National Defence and Public Works and Government 
Services Canada (PWGSC) signed decision documents for 
government consideration 


National Defence provided letter to PWGSC to justify procuring 
F-35 without competition 


Chief of the Air Staff approved formal statement of operational 
requirement 


National Defence completed formal options analysis concluding 
F-35 was only aircraft to meet requirements 


Government of Canada announced decision to buy 65 F-35s 
through JSF Program 


National Defence gave PWGSC access to final statement of 
operational requirement 


Parliamentary Budget Officer released report estimating cost of 
F-35 purchase 


: 


National Defence responded to Parliamentary Budget Officer’s 
report 


Expected events 


National Defence plans to seek Treasury Board approval 
and expenditure authority for Next Generation Fighter Capability 
project 


Beginning of transition to F-35 


b 2012 7 : 
ERG eee Oa a 
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F-35 developmental testing complete 


Second phase expected to end 


Fully capable aircraft available 
Expected delivery of first F-35s in Canada 


Estimated end of CF-18 fleet life 


Production of F-35 expected to end 


Third phase expected to end 


Source: Adapted from various National Defence, Industry Canada, Public Works and Government Services Canada, and JSF Program Office documents 
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2.25 In determining whether the federal government applied due 
diligence with respect to Canada’s participation in the JSF Program, we 
examined the extent to which key decisions, especially those taken 

in 2006, were supported by appropriate information and analyses, 
consultation, departmental oversight, and government approvals. 


Early efforts to promote industrial participation were successful 


2.26 For most defence contracts signed by the Government of 
Canada, the government’s Industrial Regional Benefits Policy applies. 
This policy seeks to ensure that defence purchases generate high-value 
business for Canadian industry and, in effect, guarantees work for 
Canadian companies, usually equivalent to the value of the acquisition 
contracts awarded. 


2.27 For the F-35, under the provisions of the JSF Program, partners 
cannot require prime contractors to provide work to companies in 
their countries. For Canada, this fact meant that the Industrial 
Regional Benefits Policy could not be applied. Industrial benefits were 
not guaranteed; rather, companies from partner countries were, and 
still are, eligible to obtain contracts, provided that the companies meet 
the “best value” criteria (including cost, schedule, and quality) and 
that the country buys the F-35 as a partner in the JSF Program. 


2.28 We examined how National Defence and Industry Canada 
carried out their respective activities and whether ministers and senior 
departmental decision makers were duly informed of the conditions on 
industrial participation established by the JSF Program. 


2.29 In 2000, National Defence gained Industry Canada’s assistance in 
supporting the JSF Program, and together they approached the potential 
prime contractors to understand their requirements and to seek 
opportunities for Canadian industry. In early 2001, an interdepartmental 
JSF team was established to coordinate the federal promotion of 
economic opportunities for Canadian industry, trying to match Canadian 
capabilities with the needs of the prime contractors. A memorandum of 
understanding was signed by National Defence, Industry Canada, Foreign 
Affairs and International Trade Canada, and the Canadian Commercial 
Corporation; this memorandum defined roles and responsibilities for each 
organization in conducting outreach missions with US contractors. 


2.30 We found that the collaborative arrangement among federal 
departments was effective. We also found early efforts to secure 
industrial opportunities through the system development and 
demonstration phase were significant and the results were successful. 
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By 2006, National Defence estimated that Canadian companies had 
received 150 contracts valued at approximately CAN$157 million. 
We also observed that, prior to signing the various MOUs in 2002 
and 2006, the ministers of Industry Canada and National Defence, 
and senior decision makers in both departments, were informed that 
industrial benefits could not be guaranteed under the JSF Program. 


Projections of industrial benefits fluctuated and uncertainties 
were not communicated 


2.31 The forecasted value of industrial benefits is significant. It has 
been used extensively as a basis for key decisions related to Canada’s 
participation in the JSF Program and the purchase of the F-35 as well 
as for communications to Canadians. Making forecasts is challenging 
and depends on, among other things, such factors as the total number 
of aircraft that will be produced eventually (which was unknown at the 
time of this audit) as well as the ability of Canadian companies to 
compete in a global market. 


2.32 
benefits. First, there are significant fluctuations in the estimates of 
contract opportunities presented by National Defence and Industry 
Canada to senior decision makers and ministers (Exhibit 2.4). For 


We have two concerns regarding the forecasting of industrial 


example, prior to signing the 2002 MOU for system development and 
demonstration, National Defence estimated between CAN$8 and 

10 billion in projected benefits. In 2006, prior to signing the MOU for 
the production, sustainment, and follow-on development phase, 
National Defence communicated estimates to ministers that ranged 
from US$5.2 to 14.7 billion. In 2009, it estimated these benefits to be as 
high as US$16.6 billion; this estimate was reduced to US$15.4 billion 
leading up to the government’s 2010 decision to purchase the F-35 jets. 


Exhibit 2.4 Projected benefits (in $billions) to Canadian industry, presented at key points to decision makers and ministers, have fluctuated widely 


National Defence briefing 
documents 


Industry Canada briefing 
documents 


CAN$8-10 


CAN$8-10 


US$15.4 US$12 


US$16.6 US$15.4 


US$14.7 


ee 


Source: National Defence and Industry Canada documents 


Unaudited figures 
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2.33 Second, we have concerns about the basis of the projections of 
industrial benefits for Canadian companies that National Defence and 
Industry Canada provided to ministers. Prior to 2006, National Defence 
took the lead in monitoring and forecasting industrial benefits, using a 
database that it designed and filled with data. In 2006, responsibility for 
monitoring and reporting was taken over by the prime contractors, in 
accordance with the industrial participation MOUs they signed with 
Industry Canada. Projections made by the prime contractors were (and 
continue to be) extrapolated over the entire production period, and 
were based on a combination of opportunities related to 


* contracts that were already awarded in the system development 
and demonstration phase, 


* potential contract opportunities that are offered exclusively to 
Canadian companies, and 


* potential contract opportunities that are available through 
competition to companies from partner countries. 


This third category represents the majority of the value of available 
opportunities, but these are the least certain, since Canadian companies 
must compete against companies from other partner countries. 


2.34 Since 2006, Industry Canada and National Defence used these 
projections as the basis of information provided to decision makers and 
ministers. These projections were not independently validated by 
federal departments, and in fact, this validation was difficult to do, 
since the data is largely proprietary. In 2010, the combination of 
estimated contract opportunities totalled US$12 billion. National 
Defence added to these projections its own estimates of benefits that 
may accrue to Canadian companies as a result of assumed expenditures 
on local sustainment over 20 years. These estimates were based on 
assumptions about the type of work that may be performed in Canada 
and how much of it National Defence may direct to Canadian 
companies. These estimates are not related to the industrial 
participation opportunities provided by the prime contractors. In 2010, 
the benefits related to sustainment were estimated to be US$3.4 billion. 


2.35 We found that briefing materials prepared by the departments for 
decision makers and ministers did not explain the basis for the 
projections, or the consequent limitations involved in relying on those 
projections for decision making. Moreover, in the majority of cases, 
only the most optimistic scenario was put forward, rather than a range 
of potential benefits that reflected the inherent uncertainties in the 
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projections. We are concerned, because these projections were used to 
support key decisions related to Canada’s participation in the JSF 
Program and the purchase of the F-35 aircraft. 


Preliminary analysis was conducted to consider various fighter jet options 


2.36 The 2006 MOU for the third phase of the JSF Program 
(production, sustainment, and follow-on development) does not commit 
partner countries to purchase the F-35 aircraft. Nonetheless, in our 
view, sound management practices would have required that National 
Defence assure itself that the F-35 was a suitable, if not preferable, 
aircraft before further committing Canada and Canadian industry to the 
JSF Program. We therefore examined departmental activities in relation 
to identifying operational requirements and assessing how available 
options would meet those requirements. 


2.37 These activities took place in 2005, when National Defence 
began an analysis of potential aircraft to replace the CF-18s. It defined 
preliminary operational requirements for a replacement and assessed 
five candidate aircraft against them, based in part on information 
obtained from site visits to various aircraft manufacturers. Four of these 
candidates were existing aircraft; the fifth, the F-35, was still being 
developed. In June 2006, National Defence summarized this options 
analysis in the Operational Requirements Concept Document (ORCD). 
It concluded with a strong preference for the F-35, stating, “It is not only 
the aircraft that best meets Canadian Forces’ requirements, with the 
longest life expectancy, but also is the most affordable.” 


2.38 The ORCD did not assess the number of aircraft required to 
meet mission expectations and commitments. National Defence 
provided the JSF Program Office with an estimated procurement 
number of 80 aircraft, which was included in the 2006 MOU and was 
the basis for calculating Canada’s financial contributions. This number 
of aircraft was an estimate based on the approximate number of 
modernized CF-18s existing at the time and was not based on an 
assessment of future operational needs. 


2.39 The ORCD conclusions were presented to senior officials within 
National Defence, including the Chief of the Air Staff and the 
Assistant Deputy Minister (Materiel). The conclusions were approved 
by the Chief of the Air Staff. These conclusions formed part of the basis 
upon which National Defence recommended to the Minister that 

the 2006 MOU be signed. In our opinion, this was an important activity 
that helped to support the government’s decision to sign the MOU. 
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Risks associated with participating in the Joint Strike Fighter Program were 
communicated to decision makers 


240 Treasury Board policies recognize that risk is an important 
element in decision making and needs to be identified, assessed, 

and communicated at key decision points. The policies also require 
measures to mitigate risks. We examined whether National Defence 
identified, assessed, and communicated risks related to Canada’s 
participation in the JSF Program before seeking expenditure authority 
from the Treasury Board. The key decision points were when National 
Defence sought expenditure authority for Canada’s financial 
contributions to the second and third phases of the JSF Program. 

We also examined the strategies that National Defence developed to 
mitigate identified risks. 


2.41 We found that National Defence identified and communicated 
risks and developed mitigation strategies for participating in the JSF 
Program. In 2001, prior to signing the MOU for the second phase of 
the JSF Program (system development and demonstration), risks such 
as the JSF Program being cancelled or running over budget were 
presented to senior decision makers in National Defence. The 
Department identified strategies to mitigate these risks, including 
imposing a ceiling for Canada’s financial contributions and negotiating 
provisions in that MOU that would allow Canada to withdraw from the 
JSF Program. National Defence identified and communicated similar 
risks and risk mitigation strategies prior to signing the 2006 MOU for 
the production, sustainment, and follow-on development phase. 


2.42 However, there is no documentation indicating how the risks 
were determined and analyzed. Consequently, we have no basis to 
determine if the risk assessments are complete and appropriate under 
the circumstances. 


There was a lack of consultation and communication on the procurement regime 


2.43 The 2006 MOU for the third phase of the JSF Program 
(production, sustainment, and follow-on development) prescribes, 

for partners, the regime for procuring the F-35. This regime has 
implications for whether and how Public Works and Government 
Services Canada (PWGSC) carries out its role as the contracting 
authority for large defence procurements. Therefore, we believe 
National Defence should have engaged PWGSC and central agencies, 
before and after signing the 2006 MOU, to map out a strategy and 


process for obtaining government approvals and managing applicable 
procurement rules. 
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2.44 This did not happen. While negotiating and finalizing the 2006 
MOU, National Defence did not consult PWGSC to determine 
whether and how the procurement process outlined in the MOU 
mechanism could be managed in accordance with Canadian laws and 
policies. The reason is that National Defence did not consider this 
situation to be a procurement; rather, it was an initiative to gain 
industrial benefits and obtain technological transfer. PWWGSC was not 
given a copy of the 2006 MOU until December 2009. 


2.45 We examined what ministers were told in briefing materials and 
submissions to government related to the decision whether to sign 
the 2006 MOU. We found that the ministers of National Defence and 
Industry Canada and those ministers on the Treasury Board were not 
fully informed in the materials about the procurement implications of 
the 2006 MOU. Briefing materials we have reviewed describe the 
features and benefits of Canada’s participation in the JSF Program, 
with particular emphasis on industrial benefits. While ministers were 
told, correctly, that signing the 2006 MOU did not commit Canada to 
buy the F-35, we did not see evidence they were told that retaining 
industrial benefits depended on buying the F-35 as a partner in the JSF 
Program. Also, while ministers were told that the 2006 MOU did not 
prevent Canada from having a competition in the future, they were 
not told of the practical limitations of doing so. For example, as a 
partner in the development of the F-35, National Defence’s long- 
standing relationship with and access to proprietary data from one of 
the prime contractors, coupled with the unique benefits offered only to 
partners, meant that other potential aircraft manufacturers would be 
disadvantaged from competing fairly. 


2.46 In summary, we found that National Defence, as the lead 
department, took the appropriate steps in managing Canada’s 
participation in the Joint Strike Fighter (JSF) Program. The 
Department managed industrial participation well (together with 
Industry Canada), identified and communicated risks and mitigation 
strategies related to participating in the Program, and assessed options 
before signing the 2006 MOU. However, National Defence did not 
fully inform decision makers of the implications of participation in the 
JSF Program for the acquisition process. In some cases, documented 
analysis did not exist to support decisions. 


2.41 As noted in the introduction, the federal process to procure 
defence equipment and to support decision making reflects legislation, 
policy, and departmental guidance. National Defence’s formal project 
management process, in place during the period of this audit, is depicted 
in Exhibit 2.5. (This process was revised in September 2011.) 
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National Defence is responsible for leading project management 
activities. Public Works and Government Services Canada (PWGSC) 
plays a critical role in contracting activities. We examined whether 
the departments applied due diligence in managing the replacement 
of the CF-18 fighter jets, called the Next Generation Fighter 
Capability project. 


2.48 Within National Defence, a capital project officially starts after it 
is formally designated as such, at the end of the Project Identification 
Phase. For the fighter jets, this occurred in January 2010, when the Next 
Generation Fighter Capability project was approved internally and the 
formal Project Options Analysis Phase was started. That said, many of 
the types of activities required under the formal process—such as 
identifying operational requirements and analyzing options, assessing 
risks, and estimating costs—had already been under way for years. 


Exhibit 2.5 Key phases and steps in project management 


Identification Phase 
Capability deficiency identified 


Project initiated 


Options Analysis Phase 
Operational requirements defined 
Total project life-cycle costs estimated 
Risks assessed 
Options analyzed 
Procurement strategy proposed 


Definition Phase 


Total project cost estimates revised 
Detailed project risk assessment conducted 
Schedule defined 


Implementation Phase 


Project implemented 
Project monitored 


Close-out Phase 


Project completed 
Lessons learned 


Source: Adapted from the project management process used within National Defence 
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At the end of the Project Options Analysis Phase, National Defence 
typically seeks formal project approval from the Treasury Board. At the 
time of this audit, National Defence expected to do so in 2012. 


Key decision-making steps and documents were of little consequence 


2.49 By the end of 2006, officials from National Defence and the 
Canadian Forces were actively involved in the development of the 
F-35. It was clearly the fighter jet of choice, and, in our view, a number 
of activities and decisions had put in motion its eventual procurement. 
By that time, National Defence had 


¢ completed a preliminary options analysis of contender aircraft and 
concluded in favour of the F-35, 


* signed three memoranda of understanding (MOUs) with its 
military allies, 


* assigned personnel to work full-time in the Joint Strike Fighter (JSF) 
Program Office and participated in the decision-making structure, 


¢ helped negotiate industrial participation memoranda with the 
F-35’s prime contractors (which were signed by Industry Canada), 


* successfully supported Canadian industry’s involvement in the 
initial manufacturing (knowing that retaining industrial benefits 
was contingent on buying the aircraft), and 


* committed about US$710 million to the JSF Program 
(based on a projected purchase of 80 aircraft) and disbursed 
CAN$176 million through contributions and support to industry. 


2.50 Despite the above, a decision by the Government of Canada to 
procure new fighter jets to replace the CF-18 was still required. This 
eventually happened in July 2010, when the government announced its 
decision to buy 65 F-35 aircraft. As described in the following 
paragraphs, we observed that in the lead-up to this announcement, 
required documents were prepared and key steps were taken out of 
sequence. Key decisions were made without required approvals or 
supporting documentation. We believe that National Defence ought to 
have treated procurement of the F-35 as a project in 2006, or shortly 
thereafter, and established an appropriate management plan. Doing this 
would have provided a framework for planning subsequent activities, 
supporting key decisions, and obtaining approvals in a timely manner. 


2.51 Government policy aims to ensure that goods and services are 
acquired in a manner that enhances access, competition, and fairness. 
It requires competition but permits a non-competitive (sole-source) 
purchase if one of four conditions exists, as set out in the 
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Capability Development Board—A board that 
assists the Chief of Force Development in 
formulating decisions, direction, and guidance 
pertaining to central force development. 


Joint Capability Requirement Board—A board 
that reviews and endorses technical and 
operational requirements of projects. 
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Government Contracts Regulations. An important step in the 
procurement process is the selection of a procurement strategy, 
essentially a decision whether to run a competition or not. The use of 
an exception to competition must be fully justified and is typically 
supported by a formal statement of operational requirement and 
options analysis. It is the responsibility of Public Works and 
Government Services Canada (PWGSC) to approve a procurement 
strategy and, more broadly, to uphold the integrity of the procurement 
process by ensuring that it adheres to all relevant legislation and policy. 
We examined how National Defence and PWGSC managed activities 
supporting the government’s 2010 decision to buy the F-35. 


2.52 In May 2008, following the release of the Canada First Defence 
Strategy, National Defence identified 14 draft high-level mandatory 
capabilities for the replacement of the CF-18s. These were very broad 
and qualitative in nature and were a precursor to the formal statement 
of operational requirement. In 2008, two departmental boards— 
National Defence’s Capability Development Board and the Joint 
Capability Requirement Board—endorsed these draft high-level 
mandatory capabilities. 


2.53 Also in 2008, National Defence undertook an options analysis of 
three contender aircraft against the high-level mandatory capabilities. 
These included the F-35, which was still under development at the 
time. This analysis concluded that, while all three aircraft could meet 
the high-level mandatory capabilities, the F-35 offered the “best value” 
because it provided “exceptional capability at the lowest cost and 
unparalleled benefits for the Canadian aerospace industry.” This 
analysis became pivotal to the decision-making process. The 
conclusions were presented to senior decision makers. There was no 
documentation supporting the analysis and conclusions. 


2.54 Between late 2008 and mid-2009, National Defence led a 
process to get a government decision to buy the F-35, partly in 
response to pressure from industry to commit to buy the F-35 in order 
to keep and increase industrial benefits. Senior departmental decision 
makers and ministers were advised that a delay in this decision would 
lead to possible losses of billions of dollars in potential contracts for 
Canadian companies. Based on this, and on the conclusions of 

the 2008 options analysis, National Defence recommended to central 
agencies, other government departments, and its Minister that Canada 
commit to buying the F-35 through the provisions of the 2006 MOU 
for the third phase of the JSF Program, without competition. National 
Defence recognized at the time that holding a balanced competition 
among contender aircraft would be “exceedingly difficult,” in part 
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according to manufacturers, incorporate the 
most modern technologies, such as stealth, 
advanced radar, and integrated avionics. There 
is no accepted or objective definition of fifth 
generation capability. 
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because of Canada’s long-standing partnership in the development of 
the F-35. Ultimately, this decision process was put on hold, but for 
reasons unrelated to holding a competition. 


2.55 In early 2010, National Defence restarted the process to obtain a 
government decision to buy the F-35. By this time, the Next Generation 
Fighter Capability Project had officially started and the Project Options 
Analysis Phase (Exhibit 2.5) was under way. It was at this time that 
PWGSC became actively involved in the decision-making process. 


2.56 Since National Defence intended to recommend the purchase 
of the F-35 without competition, it had to identify and justify an 
appropriate exception to competitive tendering set out in the 
Government Contracts Regulations. One of the permitted exceptions is 
that “the nature of the work is such that it would not be in the public 
interest to solicit bids.” Up until late May 2010, National Defence 
planned to use this exception, based in part on the conclusions from 
the 2008 options analysis and on its argument that it needed a fighter 
jet with fifth generation capability. While this exception was 
supported by PWGSC, ultimately the use of this exception was not 
supported inter-departmentally. National Defence then decided to use 
another of the permitted exceptions, that “only one person [that is, 
contractor(s)] is capable of performing the contract,” and the decision- 
making documents were reworded accordingly. 


2.57 ‘To support the use of this exception, National Defence was 
required to identify its operational requirements and to provide a full 
justification to PWGSC. Neither was provided to PWGSC in a timely 
manner, despite several requests from PWGSC. PWGSC was not 
given a copy of the statement of operational requirement until 
August 2010, well after the government had announced its decision 
to purchase the F-35 in July 2010. In fact, the statement was approved 
by the Chief of the Air Staff after decision-making documents related 
to the government’s July 2010 announcement had been signed and 
submitted for consideration by ministers. Then, according to National 
Defence, the formal options analysis was completed (as required as 
part of the Project Options Analysis Phase as shown in Exhibit 2.5), 
which concluded that the F-35 was the only available aircraft that 
could meet the mandatory requirements of the Canadian Forces. The 
conclusion was cited as the basis for the government’s decision to 
purchase the F-35 without competition. 


2.58 While decision-making documents were being finalized in late 
May 2010, PWGSC questioned National Defence’s assertion that no 
other aircraft met its mandatory requirements. Senior decision makers 
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in PWGSC were informed that PWGSC had not been provided sufficient 
justification to support National Defence’s proposed procurement 
strategy and assertion that only one company was capable of performing 
the contract. In lieu of a finalized statement of operational requirement 
or a completed options analysis, PWGSC informed National Defence on 
1 June 2010 that it would endorse the sole-source justification if National 
Defence provided a letter confirming National Defence’s requirement for 
a fifth generation fighter and confirming that the F-35 is the only such 
aircraft available. The same day, National Defence provided such a letter. 
There were no other supporting documents. By this time, decision- 
making documents had already been signed in both National Defence 
and PWGSC. It is important to note that the term “fifth generation” is 
not a description of an operational requirement. 


2.59 We found that National Defence engaged PWGSC late in the 
decision-making process and hampered PWGSC's ability to carry out 
its responsibilities as contracting authority to ensure the integrity of 
the procurement process. At the same time, PWGSC relied almost 
exclusively on assertions by National Defence and endorsed the 
procurement strategy in the absence of required documentation and 
completed analysis. We believe this has compromised an important 
control in the procurement process: independent validation of a 
proposed procurement strategy. 


2.60 In our view, many of the steps and documents used to support 
the government’s 2010 decision were of little consequence, because 
the key questions of whether to procure the F-35 and whether to run a 
competition were effectively determined by decisions taken much 
earlier, calling into question the integrity of the process. Not only were 
they of little consequence, they might also have been unnecessary if 
National Defence had sought government approval at an earlier stage 
to be completely exempt from the requirement to fit the procurement 
into one of the specified exceptions to competitive tendering. 
Practically speaking, by 2010, Canada was too involved in the JSF 
Program and the F-35 to run a fair competition. 


Risks and mitigating strategy did not reflect cost increases and delays 


2.61 ‘Treasury Board policies require risks to be identified and 
communicated to decision makers at key decision points and to be 
reassessed periodically throughout the life of the project. Our earlier 
findings focused on risks associated with Canada’s participation in the 
JSF Program. This section addresses how National Defence identified 
and communicated the risks associated with Canada’s reliance on the 


JSF Program and the F-35 to replace the CF-18. Such risks relate to 
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potential schedule slippage, cost increases, and technical challenges 
that may affect fulfillment of the mandatory operational requirements. 
We reviewed the briefing materials that were provided to various 
senior departmental committees and the Minister of National Defence 
between 2006 and 2010. We also examined the strategies developed to 
mitigate identified risks. It is important to note that the management 
of cost, schedule, and technical risks associated with the development 
and production of the F-35 is the responsibility of the US government. 
National Defence relies on the JSF Program Office to obtain 
information about such risks. 


2.62 We have concerns with the manner in which National Defence 
identified and communicated risks. As noted in the introduction to 
this chapter, the JSF Program has experienced cost increases, schedule 
delays, and technological difficulties, and has been subject to several 
major reviews. Officials from National Defence who participated in the 
senior decision-making committees of the JSF Program were regularly 
informed of these problems. Yet in briefing materials from 2006 
through 2010 that we have reviewed, neither the Minister nor decision 
makers in National Defence and central agencies were kept informed 
of these problems and the associated risks of relying on the F-35 to 
replace the CF-18. In 2009, in presentations to central agencies and 
the interdepartmental community, National Defence identified the 
risk of potential cost increases but asserted that costs had stabilized 
and that all major outstanding cost issues had been resolved. In 2010, 
when the JSF Program was undergoing a comprehensive review of cost, 
schedule, and technology development due to ongoing problems, 
neither the Senior Review Board nor the Minister was informed of 
these problems in briefing materials. According to National Defence, 
until 2010, when the decision was made to procure the F-35, none of 
the cost, schedule, or technical risks associated with the JSF Program 
involved a significant impact to Canada. 


2.63 When communicating risk information to departmental decision 
makers, National Defence provided an overall risk rating for replacing 
the CF-18 with the F-35. The Senior Review Board assessed this rating 
as “medium” prior to the July 2010 announcement. This rating is lower 
than previous ratings provided to decision makers in 2008 and was, in 
our opinion, inconsistent with the problems the JSF Program was 
experiencing at the time. There was no documented analysis to 
determine how the risks were assessed and rated. 


2.64 With respect to mitigating risks associated with potential cost 
increases, schedule delays, and technical challenges, since 2006, 
National Defence has consistently taken the position that it would 
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make its purchase at the ideal time of the production cycle. That is, it 
would do so when economies of scale are at their highest, prices are at 
their lowest, and all technical difficulties have been resolved. This 
mitigation strategy has not changed since 2006, despite the fact that 
the system development and demonstration phase was extended 

to 2018 and full-rate production is now expected to start in 2018. 

In February 2010, National Defence received formal communication 
from the US Under Secretary of Defense for Acquisition, Technology 
and Logistics that developing the F-35 would cost more and take 
longer to finish than planned and that the US Department of Defense 
was reassessing its cost projections. 


2.65 In our opinion, National Defence has been overly confident 
about potential mitigation strategies, especially given that the F-35 is 
still under development and that the CF-18 is expected to reach the 
end of its life between 2017 and 2020. The transition to the F-35 is 
expected to start in 2017 and be completed in 2020, at which time the 
CF-18 fleet would be retired and no longer supported. 


Full life-cycle costs were not presented and were likely underestimated 


2.66 Treasury Board policies require consideration of all relevant 
costs over the useful life of equipment, not just the initial acquisition 
or basic contract cost. Careful planning and full costing are needed 
to ensure that all of the elements required to provide the needed 
capability come together in a timely and predictable way and that 
adequate funds are available to support the equipment over the long 
term. We examined whether National Defence conducted full life- 
cycle costing related to its Next Generation Fighter Capability 
project and whether cost estimates were complete, supported, and 
validated, using the best information available at the time. 
Estimating future full life-cycle costs for military equipment, 
especially the F-35, is challenging. 


2.67 ‘To determine the unit price of the F-35, Canada relies in part on 
data generated by the US government. A Selected Acquisition Report 
is presented annually to the US Congress. Once approved, the JSF 
Program Office produces for Canada a “bilateral cost breakdown.” 
This breakdown contains the predicted unit recurring flyaway cost, 
based on the number of aircraft Canada plans to purchase and when it 
plans to do so. Based on the 2009 bilateral cost breakdown, the 
predicted average unit recurring flyaway cost for Canada, for 

80 conventional takeoff and landing variant aircraft, was 

US$75 million. To this cost, National Defence must add the predicted 
cost of Canadian modifications to the basic variant, which include 
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adding drag chutes for landing on short runways and adding a different 
system for refuelling in the air. The JSF Program Office also provides 
National Defence with the predicted sustainment cost of the F-35 
conventional takeoff and landing variant. National Defence reviews 
the cost breakdown for consistency with its planning assumptions. 
Following the government’s 2010 announcement, National Defence 
informed the US Joint Strike Fighter Program Office of its plan to 
reduce the purchase to 65 aircraft. 


2.68 Exhibit 2.6 illustrates two cost estimates developed by National 
Defence: the first was used for decision making in June 2010; the 
second was presented to Parliament in response to the March 2011 
report of the Parliamentary Budget Officer. It is important to note that 
some of these costs are specific to the F-35 (such as capital acquisition 
and sustainment), whereas other costs are more broadly associated 
with fighter jet capability (such as personnel). 


Exhibit 2.6 National Defence’s estimated 20-year costs for the F-35 have varied 


Aircraft 5,580 - 
gl acquisition Canadian modifications 420 ; - 
costs—aircraft 

Total capital costs for 65 F-35s 6,000 6,000 

Initial logistics and training (including simulators) 1,320 1,300 

Project management (initial) | 160 <= 200 

Weapons (initial buy) 270 300 
oils one Infrastructure 400 400 
acquisition costs : 

Contingency, inflation 830 800 

Total additional capital acquisition costs 2,980 . 3,000 

Contracted sustainment 5,710 5,700 

Contingency | 860 | S 
Personnel, : 
operating, and Operating costs 4,830 - 


maintenance costs 


National Defence personnel 


Source: National Defence 
Unaudited figures 
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2.69 ‘To date, there have been two key announcements regarding the 
budget for replacing the CF-18s: the May 2008 Canada First Defence 
Strategy and the July 2010 announcement of the F-35 purchase. The 
Canada First Defence Strategy established a budget of CAN$9 billion 
to acquire 65 next generation fighter aircraft. This budget figure was 
subsequently included in National Defence’s 2009 Investment Plan. 
It was also carried forward into decision documents to support the 
July 2010 announcement. Also in 2008, a budget of CAN$16 billion 
was established to operate and sustain the F-35 over 20 years. These 
budgets were based on estimates provided by National Defence, yet 
there is no documented analysis to show how they were developed. 
In 2010, National Defence informed decision makers that the costs of 
operating and sustaining the F-35 would be covered by existing funds. 


2.70 These budgets have since been treated as a maximum by 
National Defence, yet many decisions that could affect the ultimate 
costs are still to be taken. Moreover, as noted in paragraphs 2.11 

and 2.12, United States’ estimates of the future purchase price of the 
F-35 are in flux. Estimates for sustainment costs are not fully 
developed. Although the budgets established for the acquisition 
(CAN$9 billion) and for operations and sustainment 

(CAN$16 billion) include provision for contingency, there is a risk 
that these budgets may not be sufficient. 


2.711 We have a number of observations regarding the life-cycle 
costing for the F-35. First, costs have not been fully presented in 
relation to the life of the aircraft. The estimated life expectancy of the 
F-35 is about 8,000 flying hours, or about 36 years based on predicted 
usage. National Defence plans to operate the fleet for at least that 
long. It is able to estimate costs over 36 years. We recognize that long- 
term estimates are highly sensitive to assumptions about future costs 
as well as to currency exchange rates. However, in presenting costs to 
government decision makers and to Parliament, National Defence 
estimated life-cycle costs over 20 years. This practice understates 
operating, personnel, and sustainment costs, as well as some capital 
costs, because the time period is shorter than the aircraft’s estimated 
life expectancy. The JSF Program Office provided National Defence 
with projected sustainment costs over 36 years. 


2.12 Second, the following expected costs were not accounted for: 


* Replacement aircraft. National Defence considers 65 aircraft the 
minimum number needed to meet its training and operational 
requirements. Based on past experience, National Defence 
expects to lose aircraft in the course of normal usage. Based on 
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National Defence’s assumed attrition rate, in order to maintain 
the fleet of 65 aircraft, Canada may need to purchase up to 

14 additional aircraft over the next 36 years. National Defence 
did inform the government of the need to consider the 
requirement for attrition aircraft at a later date. The cost of 
replacement aircraft is not included in the life-cycle estimate for 
this project and will be treated as a separate project in the future. 


¢ Upgrades. It is expected that over the life of the aircraft, Canada 
will need to invest in various upgrades to the F-35 fleet, both in 
software and hardware. These costs were not known when 
the 2008 and 2010 budgets were established, but have since been 
estimated to be more than CAN$1.2 billion over 20 years. 


¢ Weapons. National Defence has currently allocated enough 
weapons for an initial stockpile to last for 45 days of conflict 
operations. National Defence assumes that it will use existing 
weapons from the CF-18s to fulfill some of its needs. It will absorb 
future weapons purchases from its operating budget, and the 
purchase of new weapons during the fleet’s life cycle will be 
addressed through future acquisition projects. 


2.73. Third, many costs are not yet reliably known or cannot yet be 
estimated. These include the basic unit recurring flyaway cost of the 
aircraft, the cost of Canadian required modifications, and the cost of 
sustainment. In addition, National Defence is still developing its 
planning assumptions for operating the fleet. This involves hundreds of 
interrelated decisions about such matters as how pilot and technician 
training will be delivered, what physical infrastructure is required and 
what portion is directly attributable to the F-35, how maintenance and 
repair activities will be supported, and what they will cost. National 
Defence currently assumes that average annual maintenance and 
repair costs for the F-35 fleet will be the same as for the CF-18 fleet. 
At the time of this audit, the project had not yet entered the formal 
Project Definition Phase, when many of these decisions will be taken 
and the cost implications will be better known. Consequently, 
estimates of the full life-cycle costs for the F-35 will likely change 

as the basis for the estimates becomes firmer. 


2.14 Also unknown are the cost implications if further delays in the 
delivery of the F-35 require National Defence to make significant 
investments to extend the operation and life of the CF- 18 fleet. 


2.15 We believe that establishing a budgetary cap without the aid of 
more complete and reliable cost estimates creates significant risks. 
National Defence may have to abandon or reduce some of the 
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Canadian Force’s capabilities, for example, by reducing the number of 
aircraft purchased or their flying hours. Alternatively, it may need to 
seek additional funds from the government or use funds from other 
parts of its capital or operating budgets. This could make it difficult to 
determine the complete life-cycle costs associated with the F-35 
fighter jet capability. 


2.16 We also have significant concerns about the completeness of cost 
information provided to parliamentarians. In March 2011, National 
Defence responded publicly to the Parliamentary Budget Officer’s 
report. This response did not include estimated operating, personnel, or 
ongoing training costs (Exhibit 2.6). Also, we observed that National 


- Defence told parliamentarians that cost data provided by US authorities 


had been validated by US experts and partner countries, which was not 
accurate at the time. At the time of its response, National Defence knew 
the costs were likely to increase but did not so inform parliamentarians. 


Recommendation 


2.77 Recommendation. National Defence should refine its estimates 
for complete costs related to the full life cycle of the F-35 capability, 
and provide complete estimated costs and the supporting assumptions 
as soon as possible. Furthermore, National Defence should regularly 
provide the actual complete costs incurred throughout the full life 
cycle of the F-35 capability. 


The Department’s response. Agreed. National Defence will continue 
to refine its full life-cycle cost estimates for the F-35 capability and 
commits to making the estimates and actual costs of the F-35 available 
to the public. 


Conclusion 


2.78 The Joint Strike Fighter (JSF) Program is unique. In this context, 
National Defence, as the lead department, exercised due diligence in 
managing Canada’s participation in the Program. National Defence 
managed industrial participation well (together with Industry Canada), 
identified and communicated risks and mitigation strategies related to 
JSF Program participation, and assessed options before signing 

the 2006 memorandum of understanding (MOU), committing Canada 
to the third phase of the JSF Program (production, sustainment, and 
follow-on development.) However, National Defence did not fully 
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inform decision makers of the implications of participation in the 
JSF Program for the acquisition process. In some cases, documented 
analysis did not exist to support decisions. 


2.19 Industry Canada exercised due diligence in managing Canada’s 
industrial participation in the JSF Program. In partnership with National 
Defence, Industry Canada worked to secure industrial participation. 


2.80 National Defence did not exercise due diligence in managing 
the process to replace the CF-18 jets. National Defence did not 
appropriately consult Public Works and Government Services Canada 
(PWGSC) on the procurement implications of the 2006 MOU for the 
third phase of the JSF Program or develop an appropriate plan for 
managing the unique aspects of the acquisition. Problems relating to 
development of the F-35 were not fully communicated to decision 
makers, and risks presented to decision makers did not reflect the 
problems the JSF Program was experiencing at the time. Full life-cycle 
costs were understated in the estimates provided to support the 
government’s 2010 decision to buy the F-35. Some costs were not fully 
provided to parliamentarians. There was a lack of timely and complete 
documentation to support the procurement strategy decision. 


2.81 PWGSC did not demonstrate due diligence in its role as the 
government’s procurement authority. Although it was not engaged by 
National Defence until late in the decision-making process, PWGSC 
relied almost exclusively on assertions by National Defence and 
endorsed the sole-source procurement strategy in the absence of 
required documentation and completed analysis. 


2.82 Both National Defence and Public Works and Government 
Services Canada disagree with our conclusion that they did not 
demonstrate due diligence in their respective roles in the replacement 
of the CF-18 jets. The departments believe that the level of due 
diligence was appropriate within the time frame covered by this audit. 


2.83 Procuring developmental equipment can bring unique risks and 
challenges. In our opinion, the experience with the acquisition of the 
F-35 has potential lessons for development and acquisition of other 
military equipment. In this context, while National Defence did 
several things well, we have described several concerns in the chapter. 
We do not believe a recommendation based on these concerns is 
required, given that best practices and policies governing these areas 
are sufficient. 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these 
standards as the minimum requirement for our audits, we also draw upon the standards and practices of 


other disciplines. 


Objectives 
The audit objectives were to determine whether 


« National Defence and Industry Canada applied due diligence in managing Canada’s participation in 


the United States (US)-led Joint Strike Fighter (JSF) Program, and 


» National Defence and Public Works and Government Services Canada (PWGSC) applied due 
diligence in managing the Canadian Next Generation Fighter Capability project for the replacement of 
the CF-18 fighter jets. 


Scope and approach 


The audit was divided into three lines of enquiry: one for National Defence and one for Industry Canada 
on Canada’s participation in the US-led JSF Program; and the third line of enquiry for National Defence’s 
and PWGSC’s management of the Canadian Next Generation Fighter Capability project. Each line of 
enquiry examined the extent to which the departments applied due diligence in managing their respective 
responsibilities. For National Defence, these responsibilities relate largely to the JSF Program and project 
management, and for Industry Canada, to industrial participation. We also examined the role of PWGSC 
in relation to its responsibilities for procurement. 


In addition to our audit examination work, we obtained background information to better understand the 
history of the JSF Program and to provide context for Canada’s participation in it. 


The audit examined documents and correspondence contained in National Defence’s program, project, 
and payment files; Industry Canada’s industrial participation and contribution files; and PWGSC’s 
procurement files. We conducted interviews with individuals who are currently or were formerly involved 
in the management of the JSF Program. We interviewed officials of the JSF Program Office, the Cost 
Assessment and Program Evaluation directorate at the US Department of Defense, the US Government 
Accountability Office, and Lockheed Martin. We visited the Canadian Forces’ Wing in Cold Lake, 
Alberta, one location of the CF-18 fleet. 


We have chosen the phrase “applied due diligence” to describe our expectation and have drawn ona 
number of sources (acts and regulations and departmental policies and guides) that address requirements 
for effective decision making, sound stewardship, and value for money. 


We did not examine certain decisions that were made, because they are policy or are beyond our technical 
capability. We did not audit the merits of the F-35 aircraft. 


We also did not audit the JSF Program or the activities of the international partners. 
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It should be noted that our conclusions about management practices and actions apply only to those of 
public servants in the federal government. The rules and regulations we refer to apply to public servants; 
they do not apply to contractors. We did not audit the records of the private sector contractors. 
Consequently, our conclusions cannot and do not pertain to the contractors’ practices or their 
performance. 


Criteria 


Our criteria reflect basic principles of good management practices and due diligence, and are based on 
relevant Treasury Board policies that support decision making and sound stewardship and contribute to 
transparency, accountability, and value for money. 


. National Defence Act 


National Defence has identified, assessed, and managed risks 
to support the decision making related to the Canadian 
participation in the Joint Strike Fighter (JSF) Program. 
(Sources: 10, 16, 17, 23) 


. Defence Production Act 


. Department of Industry Act 


National Defence has a governance and management approach 
to carry out its activities, within its mandate and authorities, 
and applied it to the Canadian participation in the JSF Program. 
(Sollrcese IZ es 4 OOS lS h6 ele lO 2 24525) 


. Financial Administration Act 
. Department of Public Works and Government Services Act 


. Government Contracts Regulations 


National Defence has carried out and sought appropriate 
oversight and approvals at key decision points related to the 
Canadian participation of the JSF Program. (Sources: 4, 10, 13, 
Se2425) 


. Contracting Policy, Treasury Board 


. Procurement Review Policy, Treasury Board 


OR COM Gl Olle ar CON ge 


. Policy on the Management of Projects, Treasury Board 


National Defence knows its contributions are being used for the 
purposes intended, in accordance with the terms and conditions 
of the 2002 system development and demonstration 
memorandum of understanding and the 2006 production, 


ph 
oO 


. Policy on Internal Control, Treasury Board 


Pak 
— 


. Policy on Transfer Payments, Treasury Board, 2008 


sustainment, and follow-on development memorandum of 12. Policy on Investment Planning—Assets and Acquired 
understanding. (Sources: 4, 10, 14, 15) Services, Treasury Board 

Industry Canada has identified, assessed, and managed risks 13. Policy on Financial Management Governance, 

and benefits related to the Canadian industrial participation in Treasury Board, 2009 

the JSF Program. (Sources: 8, 10, 16, 17, 19) 14. Account Verification Policy, Treasury Board, 1994 
LUN) Canada has sl Se peat ae and management approach 15. Policy Framework for Financial Management, Treasury 
to carry out its activities, within its mandate and authorities, Board. 2010 

and applied it to the Canadian industrial participation in the JSF 

Program. (Sources: 3, 10, 11, 13, 15, 18) 16. Policy Framework for the Management of Assets and 


; : Acquired Services, Treasury Board, 2006 
Industry Canada has carried out and sought appropriate 


oversight and approvals at key decision points related to 17. Framework for the Management of Risk, Treasury Board, 
the Canadian industrial participation in the JSF Program. 2010 


mes Ot) 18. Guide to Preparing Treasury Board Submissions, 


Treasury Board, 2007 
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Industry Canada knows that it has met its contribution 19. Guide to Costing, Treasury Board, 2008 
commitments in accordance with the terms and conditions of 


the 2002 system development and demonstration memorandum AUS SEU RE a ete) oot soda NEES TICR ETSI SEC 


. of understanding and the 2006 production, sustainment, and 21. Project Management Principles and Policies, 
follow-on development memorandum of understanding. National Defence 
(Solirces: 3, 10) 11, 15, 18) ; 
22. Supply Manual, Public Works and Government Services 
National Defence and Public Works and Government Services Canada 
Canada have identified, assessed, and managed key activities of ey 
project management to support the decision making related to 23. Procurement Administration Manual, National Defence, 


the Next Generation Fighter Capability project for the 2007 
replacement of the CF-18 fighter jets. (Sources: 2,5, 6, 7, 8,9, | 94. Project Approval Guide, National Defence, 1998 


HOWIZ IG ely lO ZO eZ Sa 4) 
: 25. Directive 7014 on Memoranda of Understanding, 
National Defence has a governance and management approach National Defence. 1098 


to carry out its activities, within its mandate and authorities, and 
applied it to the Next Generation Fighter Capability project for the 
replacement of the CF-18 fighter jets. (Sources: 1, 3, 9, 10, 12, 
1s) Way NS, alte US) a) 


National Defence and Public Works and Government Services 
Canada have carried out and sought appropriate oversight and 
approvals at key decision points related to the Next Generation 
Fighter Capability project for the replacement of the CF-18 
fighter jets. (Sources: 8, 10, 13, 24) 


Management reviewed and accepted the suitability of the criteria used in the audit. 


Period covered by the audit 


Although Canada’s participation in the United States-led Joint Strike Fighter Program began in 1997, 
the activities we audited occurred between January 2001 and July 2010. 


Audit work for this chapter was substantially completed on 30 September 2011. 


Audit team 


Assistant Auditor General: Jerome Berthelette 
Principal: John Reed 
Director: Joyce Ku 


Marie-Josée Bouffard 
Eve-Lyne Bouthillette 


Jolanta Hausner 
Robyn Meikle 
Stephanie Taylor 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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Appendix Recommendation 


The following recommendation is found in Chapter 2. The number in front of the recommendation 
indicates the paragraph where it appears in the chapter. The numbers in parentheses indicate the 
paragraphs where the topic is discussed. 


Procuring new fighter jets 


2.17 National Defence should refine Agreed. National Defence will continue to refine its full 

its estimates for complete costs related _ life-cycle cost estimates for the F-35 capability and commits 
to the full life cycle of the F-35 to making the estimates and actual costs of the F-35 available 
capability, and provide complete to the public. 


estimated costs and the supporting 
assumptions as soon as possible. 
Furthermore, National Defence should 
regularly provide the actual complete 
costs incurred throughout the full life 
cycle of the F-35 capability. 
(2.47-2.76) 
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CHAPTER 3 
Interest-Bearing Debt 


Performance audit reports 


This report presents the results of a performance audit conducted by the Office of 
the Auditor General of Canada under the authority of the Auditor General Act. 


A performance audit is an independent, objective, and systematic assessment 

of how well government is managing its activities, responsibilities, and resources. 
Audit topics are selected based on their significance. While the Office may 
comment on policy implementation in a performance audit, it does not comment 
on the merits of a policy. 


Performance audits are planned, performed, and reported in accordance with 
professional auditing standards and Office policies. They are conducted by 
qualified auditors who 


¢ establish audit objectives and criteria for the assessment of performance; 
* gather the evidence necessary to assess performance against the criteria; 
* report both positive and negative findings; 

* conclude against the established audit objectives; and 


¢ make recommendations for improvement when there are significant 
differences between criteria and assessed performance. 


Performance audits contribute to a public service that is ethical and effective 
and a government that is accountable to Parliament and Canadians. 


Table of Contents 


Main Points 


Introduction 


Who manages the debt? 
Influence of recent events on the interest-bearing debt 
Focus of the audit 


Observations and Recommendations 


Strategies for funding market debt 


The Department of Finance Canada uses a debt strategy model to support funding decisions 
Judgment and qualitative analyses inform decisions about debt strategies 
The Department has verified the soundness of the debt strategy model 


Reporting on the overall performance of the debt management strategy to the senior level 
needs further improvement 


Management of emerging risks 


A sound risk management framework is in place 
Emerging risks are monitored 
The Department has a limited strategy for nurturing relations with new investors 


Reporting information on the interest-bearing debt 


Market debt is reported clearly, but there is room for improvement 
Information on the non-market debt needs to be clarified 


Conclusion 
About the Audit 


Appendices 


A. Recent events that have had an impact on Canada’s interest-bearing debt 


B. List of recommendations 
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What we examined 
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Main Points 


Debt management refers to how the Government of Canada raises 
funds to meet its borrowing requirements, what it decides and does 
about the composition of the market debt, and how it governs these 
activities. The Government of Canada’s interest-bearing debt is made 
up of principally two sets of liabilities: market debt and public sector 
pension plan liabilities. Market debt is the part of the debt that the 
government borrows in financial markets and that is managed by the 
Department of Finance Canada. It totalled $597 billion at 

31 March 2011. Public sector pension plan liabilities are part of the 
non-market debt. These liabilities represent the government's 
obligations to the employee pension plans of the public service, 
Canadian Forces, and Royal Canadian Mounted Police (RCMP). 
They totalled $146 billion at 31 March 2011. 


Together, market debt and the pension plan liabilities make up over 
92 percent of the interest-bearing debt, which totalled $802 billion at 
31 March 2011. In 2010-11, interest charges on the debt totalled 


$30.9 billion and represented 11.4 percent of government expenses. 


We examined how the Department of Finance Canada develops 
strategies to manage market debt. We looked at its risk management 
practices and at how it monitors and reports on performance of the 
debt-funding strategy. We also examined how the Department of 
Finance Canada and the Treasury Board of Canada Secretariat report 
information about charges on the interest-bearing debt as well as the 
budgetary impact of the public sector pension plans liabilities. 


Audit work for this chapter was substantially completed on 
31 October 2011. Further details on the conduct of the audit are 
in About the Audit at the end of this chapter. 


Debt management is the federal government’s largest program (after 
transfers to seniors and to other levels of government). How market 
debt is managed has a direct impact on the government’s fiscal 
strength. Debt managers need to set appropriate market debt strategies 
using robust processes and tools—first, to balance costs and risks so 
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What we found 


that low-cost and stable funding can be raised, and second, to support 
well-functioning markets for Government of Canada securities. 


Deteriorating sovereign debt conditions in the Euro zone and in the 
United States show the importance of having sound debt strategies, as 
they provide stability and assurance to market participants. Sound debt 
strategies will also support the fiscal sustainability of public finances in 
the long term. 


In addition to market debt, public sector pension plan liabilities and 
associated debt charges have a large impact on the government’s fiscal 
situation. It is therefore important that Canadians be provided with 
clear information on that debt, the associated interest charges, and 
their impact on budget deficits or surpluses. The significant amounts 
involved can influence policy choices by limiting what the government 
can afford to do. 


« In developing its debt strategies, the Department of Finance Canada 
uses a sound process that relies on a detailed and robust debt strategy 
model and on the judgment of senior managers as well as on 
consultations with dealers and investors. The model and the 
qualitative/quantitative analyses help debt managers recommend 
preferred debt structures, and are major advances since our last audit 
12 years ago. 


The Department is monitoring and achieving the objective of a well- 
functioning market for Government of Canada securities. Until 
recently, it was using metrics that did not provide a full assessment of 
the extent to which it was achieving the objective of raising low-cost, 
stable funding for the Government of Canada by arriving at a desired 
debt structure that strikes a balance between costs and risks. 
Subsequent to the period under review, the Department started 
monitoring better the extent to which it balances costs and risks, but 
it cannot yet demonstrate that the debt strategy is achieving the low- 
cost, stable funding objective. While the Department publishes clear 
information on the market debt, it could publish more details to 
indicate why a particular debt strategy was chosen and how well the 
program is performing. 


The Department of Finance Canada has a sound risk management 
framework in place to assess and monitor emerging risks as well as 
changes in financial requirements that could affect the debt strategy. 
However, it does not promote Government of Canada securities to 
market participants as well as it could to appeal to a broader base of 
investors. 
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* Canada is a leader among member countries of the Organisation for 
Economic Co-operation and Development (OECD) in recognizing 
in its financial statements the obligations arising from public sector 
employee pension plans. In fact, very few other countries report 
these obligations on their financial statements. However, while 
complete financial information on the pension plans is available, it is 
dispersed among several reports and not presented in easy-to-read 
formats. It is therefore difficult for parliamentarians and Canadians 
to readily understand the potential impact of these liabilities on the 
budgetary balance and how they influence policy choices. In 
addition, the composition of projected interest charges reported in 
the Estimates is not clearly presented. Finally, the Department of 
Finance Canada does not have timely access to the quarterly updates 
from the Public Sector Pension Investment Board about actual 
returns on public sector pension investments, which would help the 
Department assess the impact that unforeseen fluctuations could 
have on budget surpluses or deficits. 


The entities have responded. The entities have agreed with our 
recommendations. Their detailed responses follow the 
recommendations throughout the chapter. 
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Financial requirements—The difference 
between cash coming in to the government and 
the cash going out. The government may finance 
a financial requirement by increasing market 
debt through the sale of government securities or 
by reducing its cash balances. 


Public sector pension plan liabilities—the 
government’s obligations for the three major 
pension plans it sponsors: the public service, 
Canadian Forces, and Royal Canadian Mounted 
Police plans. 
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Introduction 


3.1 The Government of Canada issues securities—for example, 
marketable bonds in either domestic or foreign currency, treasury bills, 
and Canada Savings Bonds—to fund its financial requirements. As 
well, the government has other financial liabilities that are included in 
its interest-bearing debt (such as the public sector pension plans). To 
present an idea of the size of this debt, in 2010-11 interest-bearing 
debt totalled about $802 billion. Each year, interest on this debt costs 
billions of dollars. In 2010-11, interest charges on the interest-bearing 
debt represented more than 11 percent of government expenses and 
13 percent of the government’s revenues. 


3.2 For the purposes of this report, the Government of Canada’s 
interest-bearing debt is made up of two main sets of liabilities: 


¢ market debt—debt that the Government of Canada raises in 
financial markets, including marketable bonds in either domestic 
or foreign currency, treasury bills, as well as retail debt. 


* non-market debt—principally made up of public sector pension 
plan liabilities and also including other liabilities such as 
government employee and veterans’ future benefits. 


3.3. As of 31 March 2011, the government’s market debt totalled 
about $597 billion; the debt related to public sector pension plan 
liabilities totalled $146 billion (Exhibit 3.1). Together, market debt and 
public sector pension plan liabilities added up to more than 92 percent 
of the interest-bearing debt. 


3.4 As shown in Exhibit 3.2, in 2010-11, interest charges on the 
interest-bearing debt totalled $30.9 billion. Interest charges in such 
high amounts have a major impact on the government’s surpluses or 
deficits. Marketable bonds, treasury bills, and public sector pension 
plan liabilities add up to $725.2 billion (Exhibit 3.1) and cost taxpayers 
$27.5 billion in interest charges (Exhibit 3.2). Every year, the Minister 
of Finance approves the debt management strategy, which sets out, 
among other things, the Government of Canada’s objectives, strategy, 
and plans for managing its market debt. The accounting of other 
liabilities, such as the public sector pension plan liabilities and the debt 
management strategy that is chosen for borrowing funds, will influence 
interest charges. 
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Exhibit 3.1 The composition of Canada’s interest-bearing debt for the 2010-11 fiscal year ($ billions) 


Public sector pension plans*, 146.1 
a liabilities**, 58.9 


li bills, 163.0 eral 


Foreign currency debt, 7.6 Nhs 


| ebt 
/ ——Retail debt***, 10.1 596.8 


/—Marketable bonds, 416.1 


Total: $801.8 billion 


* Pension liabilities also include $802 million for the Members of Parliament pension plan and 
$2.1 billion for the judges’ pension plan. 


** Other liabilities include the future benefits of veterans and other employees; capital lease obligation 
(amount due for long-term asset lease agreements such as office space); and a market debt value 
adjustment (which includes revaluation of cross-currency swaps, unamortized discounts, and premiums 
on market debt). 


***Retail debt means the portion of the debt held by Canadians in the form of Canada Savings Bonds and 
Canada Premium Bonds. 


Source: Public Accounts of Canada, 2010-11, Volume | 


Exhibit 3.2 The composition of the interest charges on public debt for the 2010—11 fiscal year 
($ billions) 


Public sector pension plans*, 9.7 
F y— Other liabilities**, 3.1 


_7_ Treasury bills, 1.3 
ta 
_7— Foreign currency debt, 0.2 


‘\— Retail debt, 0.2 


pce Marketable bonds, 16.5 


Total: $30.9 billion*** 


* Interest on public sector pension liabilities included $37 million for the Members of Parliament pension 
plan and $91 million for the judges’ pension plan. 


** Other liabilities include the future benefits of veterans and other employees and obligations related to 
capital leases as well as revaluation of cross-currency swaps. 


*** Total does not add up exactly due to rounding. 
Source: Public Accounts of Canada, 2010-11, volumes | and III 
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Market liquidity—State of a market that 
depends on whether there are ready and willing 
buyers and sellers of government securities in 
large quantities (if so, it is considered a liquid 
market). Securities traded in large enough 
quantities can lower borrowing costs forthe 
government. 
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Who manages the debt? 


3.5 The Department of Finance Canada is responsible for managing 
the market debt and reporting information on it. The Treasury Board 
of Canada Secretariat, the Department of Finance Canada, National 
Defence, and Public Safety Canada have responsibilities for managing 
and reporting on the public sector pension plan liabilities (Exhibit 3.3). 


Influence of recent events on the interest-bearing debt 


3.6 The debt management program is the federal government’s 
largest program (after transfers to seniors and to other levels of 
government). A key objective of this program is to meet its spending 
needs through stable, low-cost funding—that is, funding that strikes a 
balance between minimizing the level of interest charges (costs) and 
exposure to the volatility of interest charges (risks). A second key 
objective is to maintain a well-functioning market for Government of 
Canada securities, which helps to keep debt costs low and stable and 
generally benefits a wide range of participants in the domestic market. 


3.7‘ For 11 fiscal years, from 1997-98 to 2007-08, the government 
recorded budgetary surpluses. When compared to its peak in 1996-97, 
total market debt had declined by more than 17 percent by 2007-08 
(Exhibit 3.4). Because the debt of the federal government was 
declining, reducing its need to borrow through securities, a continuing 
challenge for debt managers was to ensure that enough treasury bills 
and bonds were being issued to support market liquidity. In the 
2007-08 fiscal year, debt managers had to suddenly manage rapidly 
increasing financial requirements. As seen in Exhibit 3.4, new 
government borrowing totalled $203 billion between the 2007-08 
and 2010-11 fiscal years. 


3.8 Policy decisions and events, such as the consolidation of the 


-borrowings of some Crown corporations, the turbulence of financial 


markets, and federal government stimulus, led to an increase in 
government borrowing. These events and policy decisions, together 
with changes to the public sector pension plans, had an impact on the 
interest-bearing debt and its management. Appendix A explains in 
more detail these policy decisions and events. 
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Exhibit 3.3 Roles and responsibilities in managing public debt and reporting information 


Se RRO TR RTGS TE RO NIT RIG | 


Responsible organization 
Market Debt 


Department of Finance Canada 


Under Part IV (Public Debt) of the Financial Administration Act: 


¢ Provides strategic planning and the operational management of the government’s 
borrowing. 


e Directs and oversees the policy and strategy for managing funds, submitting policy 
advice to the Minister, and preparing and publishing reports on fund management. 


Other: 


e Analyzes and reports on Canada’s budgetary situation, including interest charges 
and fiscal outlook. 


Bank of Canada 


Public Sector Pension Plan Liabilities 


Treasury Board of Canada Secretariat 


Under the Bank of Canada Act: 


e Acts as the government’s fiscal agent. As the government's fiscal agent, the Bank 
of Canada distributes Government of Canada marketable bills and bonds through 


auction to government securities distributors and customers. 


Reporting on public service pension plan: 


e Under the Public Service Superannuation Act reports to Parliament (through the 
President) once a year on the administration of the public service pension plan, 
including the amounts paid into and out of the pension accounts. 


¢ Offers general guidance to the Royal Canadian Mounted Police and National 
Defence on accounting for their pension plans. 


(The Minister of National Defence and the Minister of Public Safety table annual 
reports under the Canadian Forces Superannuation Act and the Royal Canadian 
Mounted Police Superannuation Act, respectively, on how their plans are being run.) 


Public Accounts of Canada: 


e Under the Financial Administration Act, establishes the format of the Public 
Accounts of Canada jointly with the Department of Finance Canada (through the 
President of the Treasury Board and the Minister of Finance). 


e Reports the pension liability and pension expense in the Public Accounts of 
Canada, including the disclosures in the government’s financial statements, making 
sure its accounting policies conform with Canadian generally accepted accounting 
principles for the public sector. 


¢ Ensures that actuarial valuations for all public sector pension plans are prepared as 
needed for the Public Accounts. 


Estimates: 


e Leads the preparation of the Estimates in support of the government request 
to Parliament to spend money, working with departments and agencies. 
(See paragraph 3.60.) 
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Exhibit 3.4 After a long trend of declining market debt, there was a sudden increase in the Government 
of Canada’s market debt in 2008 


Market debt: 1996-97 to 2010-11 


$ billions 

600 

500 New borrowing | 

400 

300 

200 

100 

0 
- ee ae 2 es ey e Ss ce oI - ve 
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Source: Department of Finance Canada, Debt Management Report 2010-11 


Focus of the audit 


3.9 The audit examined whether the Department of Finance 
Canada and the Treasury Board of Canada Secretariat, in keeping with 
their respective responsibilities, effectively managed the interest- 
bearing debt of the Government of Canada. This audit examined debt 
management and reporting practices between 1 April 2007 and 

31 March 2011. The audit focused on the interest-bearing debt, 
including how market debt is managed and how the impact of the 
public sector pension liabilities (public service, Royal Canadian 
Mounted Police (RCMP), and Canadian Forces, including the 


Reserve) is monitored and reported. 


3.10 We audited parts of the interest-bearing debt that include 
domestic marketable bonds, treasury bills, and public sector pension 
plan liabilities. We did not examine the retail debt program, foreign 
currency borrowings, or the consolidation of borrowings by Crown 
corporations. We also left out liabilities linked with the pension plans 
of federal judges and members of Parliament because they follow 
different regimes, as well as other liabilities such as capital lease 
obligations, and the future benefits of veterans and other employees. 
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Strategies for funding market debt 


Debt structure—Combination of types and 
terms of securities that the government issues, 
including, for example, 30-year bonds or short- 
term treasury bills. 


Debt rollover—Refers to the renewal of a 
portion of the market debt that is arriving at 
maturity at a specific date and needs to be 
refinanced. 


The audit scope represents about 90 percent of the total interest- 
bearing debt. 


3.11 We looked at the processes and tools the Department uses to 
support and develop strategies for market debt. We also looked at how 
the Department manages risks and how the performance of the 
strategy for debt management is monitored, as well as how 
performance results are used to support debt management decisions, 
including the results achieved so far. Finally, we looked at how the 
Department and the Secretariat report information on public sector 
pension plan liabilities, the interests on these liabilities, and their 
budgetary impact on the government’s fiscal position. 


3.12 More details about the audit objectives, scope, approach, and 
criteria are in About the Audit at the end of this chapter. 


Observations and Recommendations 


3.13 Choosing a financing strategy for managing debt that meets 
program objectives is a challenge when fiscal and economic outlooks 
are uncertain and financial markets are volatile. Debt managers need 
to balance costs and risks in order to meet the government's objectives 
of raising low-cost, stable funding and helping to support well- 
functioning markets for Government of Canada’s securities. To meet 
these objectives, debt managers need to gradually move toward a debt 
structure that, in the long term, will minimize interest charges subject 
to managing debt rollover over time and providing liquidity across 
different maturity sectors. 


3.14 Both quantitative information and the judgment of debt 
managers need to contribute to good strategy for managing debt. Fiscal 
and economic outlooks, debt management strategy analyses, and input 
from market participants (distributors and dealers buying and trading 
securities) gathered through consultations should guide debt managers 
to find a balance between the level of interest charges (costs) and the 
volatility of interest charges (risks), and to maintain well-functioning 
markets for government securities. 


3.15 The government can choose from various financing strategies to 
meet its program objectives. When issuing securities, the government 
must consider the best funding options—securities can be issued in a 
variety of options, such as 3-, 6-, and 12-month treasury bills and 

2-, 3-, 5-, 10-, and 30-year bonds (including 30-year inflation-adjusted 
bonds). Debt managers recommend which options to issue after 
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considering borrowing costs, stability of interest charges, refinancing 
risks, and impact on domestic securities markets. Normally, issuing 
long-term securities to fund debt costs more, but it makes future 
borrowing costs more predictable and reduces refinancing risks. 
Conversely, short-term debt is less costly, but it increases the volatility 
of interest charges and the risk that the borrower could be exposed to 
potential refinancing difficulties. Debt managers must make trade-offs 
between low-cost, short-term financing and longer-term, more costly, 
but less risky financing. 


3.16 Using indicators (metrics) for monitoring and reporting the 
performance of the debt management strategy is key to managing debt 
effectively. International organizations such as the World Bank and the 
Organisation for Economic Co-operation and Development (OECD) 
recommend the regular assessment of the performance of a debt 
management strategy. Monitoring and reporting on indicators assure 
managers that the results are in line with the government’s objectives 
for managing debt. 


3.17 We examined the decision-making process that is in place at the 
Department of Finance Canada to support market debt strategies and 
policy recommendations, including how the Department’s debt 
strategy model, a computer program that simulates different financing 
strategies, is used to support debt managers in meeting the financial 
needs of the Government of Canada. We looked at whether the debt 
managers are ensuring that the model is producing reliable results. 
And, we looked at whether the Department uses indicators to assess 
and report on the performance of the debt management strategy. We 
looked at research documents, presentations, and minutes of meetings 
where senior managers discussed and analyzed debt strategies. We also 
interviewed officials from the Department and the Bank of Canada, as 
well as debt management officials in some foreign jurisdictions. 


The Department of Finance Canada uses a debt strategy model to support funding 
decisions 


3.18 We found that the Department has access to much more 
sophisticated tools and methods of analysis than were available 

12 years ago, when we last performed an audit of market debt 
management practices. The Department has put in place a sound 
process that relies heavily on the use of quantitative analyses to 
develop debt management strategies and establish trade-offs when 
balancing program objectives. The Department and the Bank of 
Canada have developed a detailed and complex debt strategy model 
(Exhibit 3.5). Work on that model started in 2002, and major 
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Medium-term debt strategy—the 
government's strategy plan for issuing securities 
over a horizon of three to five years to achieve a 
desired composition of the government debt. 


enhancements were made in 2007. Using assumptions about the 
changing economy, interest rates, and financial requirements, the 
model is made of more than 250 subroutines and over 100 parameters 
to project various mixes of the debt structure. 


Exhibit 3.5 The debt-strategy model helps debt managers develop a medium-term debt strategy 


The model 


The debt-strategy model’s results allow debt managers to identify the relationship 
between risks (volatility) and interest charges on debt (cost) by generating a graphical 
representation known as an “efficient frontier.” 


The model includes sets of equations that compute, assuming a given debt 
management strategy and future macroeconomic and interest-rate outcomes, how the 
maturing debt and new financial requirements are refinanced, how interest charges are 
calculated, and how these outcomes affect the size and makeup of the debt. 


Developing a medium-term debt strategy 


Debt managers use the model results to recommend a debt structure that supports the 
desired cost versus risk trade-off while taking into account rollover risk, maturity 
profile, and liquidity requirements over the medium term. 


The graph below shows that while relying on long-term bonds will lead to higher costs 
(see point A), it should also result in reduced volatility. Conversely, lower debt costs 
can be achieved if more short-term securities are issued, such as treasury bills (see 
point B). The short-term approach will, however, lead to higher volatility. The model 
helps debt managers identify a zone where the cost and risks are best balanced in 
order to raise low-cost and stable funding. 


Efficient frontier that summarizes the relationship between costs and risks 


@ A (mainly 30-year bonds—long-term borrowing) 


Debt cost ($) 


Zone of the preferred debt structure 


B (mainly treasury bills— 
short-term borrowing) 
ae ee . @-:-: 


Risk ($) 


3.19 The model makes it possible for debt managers to understand the 
complex relationship between key economic outcomes and helps them 
identify a desirable funding mix. The medium-term debt strategy 
created in 2010-11 was supported by analyses and results of the 2007 
debt strategy model. 
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Rollover risk—The risk that debt will have to be 
refinanced at higher cost or, in extreme cases, 
cannot be refinanced at all. 
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Judgment and qualitative analyses inform decisions about debt strategies 


3.20 We found that the Department can set the model’s parameters so 
that all funding options are retained, which is both prudent and 
essential for making funding flexible and adaptable to changing 
funding requirements, as was the case in 2008. Parameters are set to 
address issues such as 


* identifying the minimum amount of government securities 
(minimum benchmark sizes for each sector) that should be issued, 
using comments from market participants and the judgment of 
debt managers; 


* establishing the premium that the government would pay (also 
known as the penalty function or liquidity premium) if the 
amount of securities issued is not meeting market expectations 
(this parameter relies on debt managers’ judgment and is adjusted 
with feedback from market specialists); and 


* determining which mix of maturities should be used to minimize 
refinancing risk and to spread out those maturities over the course 
of a year. 


3.21 The model’s recent results show the advantage of issuing more 
short- and medium-term bonds rather than issuing long-term bonds. 
The model shows that such strategies, while improving the debt 
structure in the long run, would also reduce risks of increased interest 
charges. Using simulation results and their own judgment, debt 
managers can compare cost and risks to identify a set of possible 
funding strategies to recommend to senior management and the 
Minister. The recommendation also takes into account exposure to 
rollover risk, maturity profile, and liquidity requirements. 


3.22 We found that debt managers present to senior management 
assumptions about interest rates, the macroeconomic environment, 
and fiscal planning. The fiscal outlook that the Department uses to 
plan the debt management strategy is in keeping with the projections 
used in the government’s annual Budget (including projected 
budgetary deficits or surpluses and financial requirements). Debt 
managers hold discussions with officials from the Economic and Fiscal 
Policy Branch at the Department of Finance Canada on interest rates 
scenarios, budgetary volatility, and risk tolerance levels. Also, officials 
consider the fiscal situation and financial requirements when they 
review the proposed debt management strategy. Financial 
requirements are updated twice a year and reviewed monthly. Debt 
managers consider the uncertainty around these requirements. 


Report of the Auditor General of Canada—Spring 2012 


Chapter 3 | 13 


INTEREST-BEARING DEBT 


Government securities auction—A process 
used for selling Government of Canada debt 
securities (mostly marketable bonds and 
treasury bills) in which issues are sold by public 
tender to financial institutions authorized to bid. 


3.23 When selecting the most favourable debt portfolio, senior 
management takes into account risk preferences, risk tolerance, and 
constraints the government is facing when issuing securities. The 
constraints include the minimum and maximum dollar amount of 
securities to issue for a well-functioning market. In our review, we 
noted that senior managers consider these factors when choosing the 
best debt management strategy, and use judgment when 
recommending this strategy to the Minister of Finance. 


3.24. Active market participation is needed for government securities 
auctions to be successful and to meet the objectives of the debt 
management program. We found that feedback from market 
participants is a key part of the decision-making process for 
recommending options for the debt management strategy. 


3.25 The government follows a strategic approach for its domestic 
borrowing. Debt managers build a relationship with market 
participants based on the government’s credibility and predictability 
(for example, through transparent operations and a pre-announced 
calendar for issuing securities). Over the long term, this approach 
could lead to lower borrowing costs, lower risks, and well-functioning 
markets for government securities. The financial crisis of 2008 and the 
more recent deterioration in sovereign debt conditions in the Euro 
zone and in the United States as the audit was being completed show 
the need for sound debt strategies, as they provide stability and 
assurance to market participants. 


3.26 Discussions on the different funding options allow market 
participants to express their views and concerns over the debt 
management strategy. Regular consultations with market participants 
cover general market conditions, explore how effective the borrowing 
program is, and validate the government’s modelling assumptions, such 
as the minimum size of securities to be issued in dollars. Debt managers 
take into account the views of market participants but also rely on 
their judgement and on the results of the model to come to a final 
decision. For example, we found that debt managers discussed re- 
issuing the 3-year bond with market participants, who expressed some 
concerns about the re-introduction of the 3-year bond after a 12-year 
gap. Debt managers used their judgment as well as the results of the 
model to recommended re-issuing the 3-year bond. In another 
example, market participants expressed a preference for the 
government to increase the amount of 30-year bonds it issues, but debt 
managers recommended against it because it would have been more 
costly over the long term. 
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Sensitivity analysis—tn the context of testing 
the reliability of the debt strategy model, an 
analysis that determines how different 
assumptions will affect the results the model 
produces. 


Stress test—A simulation technique used on a 
model or a system to find out its reactions to 
different, sometimes hypothetical and extreme, 
scenarios. 
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The Department has verified the soundness of the debt strategy model 


3.27. As the model’s simulation results support the process for 
choosing a strategy, debt managers must ensure that the model is 

well designed and comprehensive in its calculations and simulations. 
Different approaches to modelling will lead to different forecasts of key 
risks and cost characteristics for a given financing strategy. An 
important step in testing the soundness of the model is doing a 
sensitivity analysis of the results to ensure that the model is reliable. 


3.28 Several key parameters must be assessed before the model’s 
results can be used to support decisions on the debt management 
strategy. We found that sensitivity analysis has been done on the 
model. This analysis showed that the modelling results were 
appropriate for informing debt managers on the best debt management 
strategy. We also found that model results were tested by applying 
potential optimistic and pessimistic economic scenarios (stress tests). 


3.29 We found that the Department exercised due diligence in 
reviewing the model’s variables and assumptions. For example, 
in 2007, the Department hired a specialist to assess the model. 
The specialist recommended changes to the model to improve its 
functionality, and those changes were made. 


3.30 We found that the Department of Finance Canada and the Bank 
of Canada are sharing information and working with other sovereign 
debt model specialists to constantly improve the government’s debt 
strategy model. Notably, a foreign debt management office is 
considering implementing the Canadian debt strategy model. 


Reporting on the overall performance of the debt management strategy to the senior 
level needs further improvement 


3.31 Debt managers need to ensure that the debt management 
strategy’s performance meets debt management program objectives 
and the expected results of the strategy. For many years, the main 
indicator that debt managers used to make sure that the government 
had access to stable, low-cost capital was the percentage of fixed-rate 
debt in the portfolio. The Department concluded that the fixed-rate 
share alone did not give enough information about how the debt 
portfolio was performing compared to the objectives of the debt 
management strategy. 


3.32 We found that the Department does monitor the debt portfolio. 
The Department and the Bank of Canada developed a new quarterly 
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Tail—the difference between the highest 
accepted yield and the average yield of the 
auction. Tail below 1 basis point reflects high 
liquidity of debt securities and is considered an 
indication of the overall effectiveness of 
auctions. 


Yield—tThe discount rate that makes the market 
price of a government security equal to its 
discounted present value. 


Auction coverage ratio (also known as “bid- 
cover ratio” )—The total amount of bids received 
divided by the dollar value of the securities that 
were auctioned. A large ratio tends to be 
associated with a large demand and therefore 
should result in a lower average auction yield. 
For example, a ratio of 2 means that bids from 
market participants were two times higher than 
the dollar amount offered by the government. 


debt management report in March 2011. The report monitors the debt 
portfolio and provides senior management with information such as 


« the government’s financial requirements; 
* outstanding debt by types and terms of securities; 
* upcoming maturities; and 


¢ the results of recent securities auctions, which are monitored to 
measure how well the market reacted to the auction. 


3.33 In this quarterly report, the Department uses a set of indicators 
that includes 


* average term to maturity; 


* refixing share (debt that is being refinanced within the next year 
as a percentage of the debt portfolio); 


¢ bonds maturing; 
* total outstanding market debt; and- 


* other selected indicators (for example, tail, yield, and auction 
coverage ratios from securities auctions). 


3.34. The quarterly report allows officials to monitor how well the debt 
management strategy is meeting the objective of maintaining a well- 
functioning market for Government of Canada securities. This 
information allows debt managers to assess how well the market for 
Government of Canada securities functions and the success of the 
medium-term debt strategy. Exhibit 3.6 shows the very low borrowing 
costs (low yield), high auction coverage, and low tail results for 
selected Government of Canada bonds issued just before 

31 March 2011. The Department of Finance Canada is therefore 
monitoring and achieving the well-functioning market objective. 


3.35 During the period under review, debt managers were using 
metrics such as the fixed-rate ratio—the ratio of fixed-rate debt to the 
total debt. These metrics did not provide a full assessment of the 
extent to which the debt management strategy was meeting the 
objective of raising stable and low-cost funding by striking a balance 
between costs and risks associated with the debt structure. Senior 
management was therefore not getting reports to track the 
performance of the debt management strategy. Without this 
information, debt managers and senior management did not know if 


the strategy was successfully meeting the stable and low-cost funding 
objective. 
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Exhibit 3.6 Selected Government of Canada securities auctions were favourable* 


5 eat 9 March 2011 1.867 2.690 0.10 
3 year 23 March 2011 2022 | 2550 0.33 
5 year 2 March 2011 2.700 2.315 0.37 
1Oyesh 2 February 2011 0.66 
30 ee 17 November; 2011 3.638 2.859 0.25 


*Low yield, high auction coverage, and low tail results bring about favourable conditions. 
Source: Bank of Canada 


3.36 Subsequent to the period under review, the Department of 
Finance Canada started to monitor the actual debt structure against 
the desired debt structure on the efficient frontier, the graph that 
summarizes the relation between cost and risks, and to report results to 
senior management. With this information, management will be able 
to assess how the actual debt structure compares to the planned 
medium-term debt strategy and can adjust that strategy if needed. 
While the objective of raising stable and low-cost funding is now better 
monitored, the Department of Finance Canada has yet to demonstrate 
that the debt structure is achieving the expected results because it just 
started monitoring the progress of the debt management strategy 
toward the preferred debt structure, and little data is available. We 
encourage the Department to pursue its efforts in assessing the key 
objective of raising low-cost stable funding and in reporting the 
performance results to senior management. 


Management of emerging risks 3.37 Unforeseen events could have major financial implications for 
the government, such as higher interest charges, refinancing 
difficulties, or larger cash requirements. For these reasons, debt 
managers must effectively manage emerging risks. International best 
practices on public debt, such as guidance by the International 
Monetary Fund and the Organisation for Economic Co-operation and 
Development, recommend risk management activities such as 
identifying and assessing risks, measuring and monitoring the impact of 
risks, mitigating and controlling risks, and finally, regularly reporting 
on risk issues. 


3.38 We examined whether the Department of Finance Canada has a 
sound framework for risk management to identify, measure, monitor, 
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and report on new risks that could have an impact on the debt 
management strategy. We examined how emerging risks are identified 
and reported to senior management and how the strategy is updated to 
mitigate emerging risks, such as sudden changes in financial 
requirements or market conditions. We reviewed the documents that 
were presented to senior management and interviewed Department 
officials and selected stakeholders. 


A sound risk management framework is in place 


3.39 As noted in this chapter, the debt strategy model factors in the 
impact of interest rate risk, rollover risks, and budgetary volatility 
when computing results. These risks are taken into account when debt 
managers target the most desirable debt management strategy. 


3.40 We found that debt strategies also include risk analyses and 
contingency plans. Managers consider both pessimistic and optimistic 
scenarios. For example, we found that the issuing of treasury bills can 
be adjusted rapidly if the government needs to borrow more funds than 
planned for in the debt management strategy. At the peak of the 
financial turmoil in the 2008-09 fiscal year, the Department relied 
heavily on treasury bills to fund new government programs and 
growing financial requirements. In the 2008—09 fiscal year, borrowing 
through the sale of treasury bills grew by $75.5 billion to $192.5 billion. 
Borrowings through bond auctions can also be adjusted to reflect 
changing circumstances, but consultations with market participants 
may be required. 


3.41 We found that the Department’s risk management framework 
allows debt managers to assess, monitor, mitigate, and report risks. 
Senior management looks at how to fund financial requirements based 
on different economic growth scenarios. We also found that the 
Department makes projections of financial requirements for the next 
10 years and considers different debt strategies. 


3.42 Many events can affect the planned financial requirements. 

In 2008-09 and 2009-10, during the financial crisis and because of 
rapidly changing financial requirements, the Department reviewed the 
debt management strategy during the year. The reintroduction of the 
three-year bond in 2009 is another example of how debt managers 
adapted the strategy to fund growing financial requirements. In 
2010-11, the Department gave a mid-year update to the Minister as 
part of the process for fall market consultations. Mid-year updates on 
the debt management strategy are provided to the Minister of Finance 
and explain why there is a need or not for a new strategy. 
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3.43. The Government of Canada holds liquid financial assets, such as 
cash deposits, to safeguard its ability to meet payment obligations in 
situations where normal access to funding markets may be disrupted or 
delayed. The government's overall liquidity levels are intended to 
cover at least one month of the net projected cash flows, including 
interest to be paid and debt refinancing needs. According to the Debt 
Management Strategy 2011-12, to meet these liquidity levels by 
2013-14, government cash assets and foreign denominated financial 
assets will need to grow by an additional $35 billion. This increased 
level of liquidity will improve the government’s ability to meet 
payment obligations should normal access to funding markets be 
disrupted or delayed. It will also support investor confidence in 
Canadian government debt. 


Emerging risks are monitored 


3.44 ‘To enable senior management to discuss risk issues, the 
government has formed committees under a governance framework for 
funds management (Exhibit 3.7). The Risk Committee, which advises 
the Funds Management Committee, reviews and gives opinions on the 
tisk implications of market and operational developments and on 
policy recommendations. The Risk Committee meets quarterly. It is 


Exhibit 3.7 The Governance Framework manages risks 


Minister of Finance 


Funds Management Committee 


A decision-making committee that advises the Minister on strategies for 
funds and risk management, directs the implementation of the approved debt 
strategy and plans, and reviews performance reports. 


Risk Committee 
Reviews and gives opinions on the risk implications 
of market and operational developments and on 
debt strategy recommendations. 


Financial Risk Office 
Advises the Risk Committee on risk issues related 
to proposals and recommendations for the debt 

strategy as well as analyzes financial and 
operational risk exposures. 


Supporting committees and working groups 
Coordinate work on funding and investment activities at a 
strategic level. 
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made up of senior officials from the Department and the Bank of 
Canada, as well as an external member. The external member was 
added in April 2010 for a two-year term to improve knowledge of 
ongoing developments in best practices for risk management and to 
provide better oversight. Because the Department and the Bank of 
Canada have different responsibilities, we found that the 

two organizations often have different views on debt issues. These 
different perspectives can benefit risk management. For example, 
before the Funds Management Committee submits a debt strategy 
proposal to the Minister for approval, Bank of Canada and 
Department officials thoroughly debate it and consider all aspects of it. 


3.45 The Financial Risk Office advises the Risk Committee on risk 
issues related to policy proposals and recommendations and analyzes 
financial and operational risk exposures. The Office is staffed by 
officials from the Bank of Canada and is independent from funds 
management operations at the Bank. 


3.46 The Funds Management Committee is made up of senior officials 
from the Department and the Bank of Canada. This decision-making 
committee’s mandate is to advise the Minister on policy and strategy 
for funds and risk management, direct the implementation of approved 
strategy and plans, and review performance reports. We found that as 
part of its annual recommendation for the debt management strategy, 
which it submits to the Funds Management Committee and the 
Minister for approval, the Department identifies and analyzes selected 
risks that affect the domestic borrowing plan. 


3.47. There are some discussions about domestic issues, such as debt 
strategy modelling, the retail debt, and borrowing by Crown 
corporations. However, we found that discussions at the Risk 
Committee were focused mainly on risks related to liquidity, foreign 
currency borrowing, and foreign financial assets, including sovereign 
debt issues. Discussions also focused on immediate emerging issues. 
As we were completing this audit, the Department provided certain 
simulation results showing that debt managers consider long-term 
fiscal sustainability. We encourage the Department to pursue its efforts 
in analyzing how fiscal sustainability issues could affect debt strategies 
in the future. 


The Department has a limited strategy for nurturing relations with new investors 


3.48 Recent events abroad have proven that loss of confidence and a 
weakening fiscal position have a major impact on borrowers. In a crisis, 
such as a severe economic downturn or a natural disaster, having a 
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diversified base of investors allows debt managers to distribute 
government securities more easily and reduce refinancing risks. In this 
context, encouraging more competition among market participants 
would lead to wider distribution of government securities, which 
reduces the financing cost for the government. Therefore, a key 
element in managing risk for domestic debt is nurturing and 
maintaining relations with market participants. 


3.49 Because communication between debt managers and investors is 
important, some international organizations have created guidelines 
on this subject. The Institute of International Finance lists a number of 
best practices for communicating with investors, including the 
following: 


¢ Staff responsible for investor relations should be identifiable and 
reachable through the government’s website. 


¢ Archives of presentations and conference call materials directed 
at market participants should be available on the website. 


* Macroeconomic data should be presented in a user-friendly 
format. 


¢ Market participants should be able to subscribe to the 
government’s website. 


The Ontario Financing Authority carries out the practices listed 
above. Brazil and France are among sovereign issuers of securities that 
make promotional presentations to other countries and provide 
macroeconomic data on their websites. Australia actively promotes its 
government securities by making presentations to investors abroad. 


3.50 We found that the Department of Finance Canada and the Bank 
of Canada use a mix of communication approaches to consult with 
market participants on debt strategy matters (such as bilateral visits, 
multilateral discussions, and the annual consultation process). We 
found, however, that the information the Government of Canada 
provided to potential new market participants is limited. Economic, 
financial, and government borrowing information is found in different 
documents and websites. Also, there are no presentations available 
online to actively promote Government of Canada securities to 
potential investors. We believe this limited effort in reaching out and 
informing investors does not help Canada to build, diversify, and 
broaden its base of investors. If there were a sudden increase in 
financial requirements that needed to be funded abroad, Canada might 
not be well-positioned to attract this funding. 
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Reporting information on the 
interest-bearing debt 
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3.51 Recommendation. The Department of Finance Canada should 
provide better financial information to market participants and expand 
its communication and marketing tools, as well as its promotional 
activities in order to reach a broader base of investors. 


The Department’s response. Agreed. The Department of Finance 
Canada’s website contains a wide range of materials on Canada’s 
borrowing programs, debt strategy, and economic and fiscal 
performance. The Bank of Canada’s website also contains details on 
Canada’s debt auctions. That said, the Department and the Bank plan 
to expand the range of economic, financial, and borrowing information 
available on their websites, provide better links between the two sites, 
and improve the organization of this information to make it more easily 
accessible. The Department recognizes the importance of maintaining 
and building on Canada’s strong reputation in global capital markets 
and will take steps to increase its engagement with domestic and 
international investors as opportunities present themselves. 


In terms of expanding promotional activities, the Department would 
note that the investor base for Government of Canada securities has 
grown in recent years, that debt auctions have been well covered, and 
that Canada’s global foreign-currency bond issues have met with very 
strong demand from international investors. Devoting significantly 
more resources to investor relations and associated promotional 
activities would not provide value for money. 


3.52 Reporting clear and complete information on the interest- 
bearing debt allows for better accountability. This reporting includes 
comparing the planned debt management strategy with actual results 
and providing information on the budgetary impact of government 
liabilities. Budget documentation and fiscal reports should cover all of 
the government’s budgetary activities (such as transfer payments, 
operating expenditures, and interest charges on the public debt) as 
well as non-budgetary ones (for example, loans and investments). The 
International Monetary Fund and the Organisation for Economic Co- 
operation and Development (OECD) have recommended that fiscal 
information be presented in a way that facilitates policy analyses and 
promotes accountability. 


3.53 We examined whether the Government of Canada reported 
complete and clear information on the market debt and the public 
sector pension plan liabilities. We reviewed the public documentation 
that reported information on the interest-bearing debt, including 
public sector pension plan liabilities, and we reviewed briefing material 
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and minutes from senior management committee meetings. We 
interviewed officials from the Department of Finance Canada, the 
Treasury Board of Canada Secretariat, and the Office of the Chief 
Actuary. Finally, we reviewed how four countries rated very high by 
credit rating agencies (Denmark, Finland, France, and United 
Kingdom) report their interest-bearing debt. 


Market debt is reported clearly, but there is room for improvement 


3.54 Parliamentarians and Canadians can read the Debt Management 
Strategy, which is a report annexed to the Budget early each year, and 
the Debt Management Report late in the fall, to better understand 
market debt issues. These reports contain information on market debt 
management, including the government’s objectives, planned 
borrowing activities, and the medium-term debt strategy. The reports 
also include several indicators, such as how many domestic marketable 
bonds and treasury bills the government has issued, the dollar amount 
issued per securities (for example, 2-, 3-, and 5-year terms), the 
composition of the market debt, the refixing share of debt, and the 
average term to maturity. 


3.55 Compared to other governments (such as Denmark and the 
United Kingdom), the Government of Canada gives little analysis to 
explain the chosen debt management strategy. For example, the 
Department of Finance Canada does not publish the results of the debt 
strategy model and information to understand funding choices and 
why a particular strategy was chosen. This information would allow 
parliamentarians and Canadians to better understand the 
government’s debt management strategy. For instance, it could explain 
why the government believes that short-term borrowing (such as using 
treasury bills), while less expensive, is not desirable because of the 
impact on rollover risk or why locking-in at relatively low interest rates 
for 30 years in order to provide stable interest charges in the long term 
is too costly. 


3.56 As explained before, while performance information on the well- 
functioning market objective is presented to senior management, the 
Department has recently developed new metrics to monitor the low- 
cost stable funding objective, and it is reporting these to senior levels. 
We believe that the Department needs to better disclose performance 
results of the debt management program in the reports it publishes. We 
found that the Department is reviewing ways to improve the 
information it publishes and is looking at how it presents and supports 
its debt management strategy in its reports. The Department has also 
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said it will adapt its debt reporting practices, as needed, to be more 
transparent. 


3.57 Recommendation. The Department of Finance Canada should 
improve its reports on the market debt by including the analysis that 
supports the debt management strategy and information on the overall 
performance results against planned outcomes. These improvements 
will help better inform parliamentarians, Canadians, and investors on 
the actual results of the debt management strategy. 


The Department’s response. Agreed. Where feasible, the 
Department of Finance Canada will aim to improve the information 
content of our reports. 


We would note, however, that compared to its peers, Canada is already 
among the most informative in terms of its debt management 
reporting. The Debt Management Strategy and Debt Management 
Report are among the most transparent documents of the 

G20 countries with respect to describing the government’s debt 
strategy as well as providing performance outcomes. New metrics have 
been added to recent reports. The Department also publishes external 
evaluations of debt management programs and operations conducted 
under the Treasury Evaluation Program. Additionally, a number of 
papers on Canada’s stochastic debt strategy model are available on the 
Bank of Canada’s website. 


Information on the non-market debt needs to be clarified 


3.58 As of 31 March 2011, the liability of public sector pension plans, 
which is part of the government’s gross debt, totalled $146 billion. In 
other words, about 18 percent of the $802 billion in interest-bearing 
debt was related to public sector pension plans. In 2010-11, around 
$9.7 billion of the $30.9 billion in interest charges was related to the 
interest expense on the public sector pension plans (Exhibit 3.1). It is 
important to monitor and report on how these liabilities and interest 
charges could affect the government’s fiscal situation. 


3.59 Reporting on public sector employee pension plans. The 
government sponsors defined benefit pension plans for its employees, 
notably the employees of the public service, the Canadian Forces 
(including the Reserve Force), and the Royal Canadian Mounted 
Police (RCMP). The government has a statutory obligation to pay 
benefits relating to these pension plans. Canada is a leader among 
member countries of the OECD in recognizing in its financial 
statements the obligations arising from the public sector employees’ 
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Estimates—Documents prepared to support 
appropriation acts. Expenditures made by 
government require the authority of Parliament. 
That authority is provided in two ways: annual 
appropriation acts that specify the amounts and 
broad purposes for which funds can be spent, 
and other specific statutes that authorize 
payments and set out the amounts and time 
periods for those payments. The Estimates 
provide additional information on voted amounts 
included in the appropriation act. Forecasts of 
statutory amounts, including interest charges, 
are also presented to give a more complete 
picture of total expenditure authorities for use 
during the fiscal year. 


Superannuation accounts—Pension funds 
established to record transactions relating to 
service accrued by members of the public 
service, Canadian Forces, or RCMP pension 
plans before 1 April 2000. In essence, the 
superannuation accounts are legislated ledgers 
and do not hold assets. 


INTEREST-BEARING DEBT 


pension plans. Very few countries report public sector employee 
pension obligations on their financial statements. Most European 
countries do not report them. In order to ensure comparability among 
countries, the OECD excludes these liabilities from the debt to be 
reported in the governments’ finance statistics. 


3.60 Reporting interest charges in the Estimates. Parliamentarians 
rely on several sources of information to review government 
operations. For information on projected government expenditures, 
they mainly use the Estimates documents, because Parliament has a 
fundamental duty to scrutinize and approve expenditures to be voted 
on. Each year, the government prepares the Estimates in support of its 
request to Parliament for authority to spend public money. These 
Estimates are tabled in the House of Commons and include 
information on voted expenditures and statutory expenditures. 
Voted expenditures are amounts included in an appropriation bill to 
be authorized by Parliament. Statutory expenditures are amounts for 
which parliamentary authority has already been obtained through 
other legislation and is presented in the Estimates for information 
purposes only. According to the Main Estimates 2011-12, statutory 
expenditures on “interests and other costs” could total $30.2 billion. 


3.61 “Interest and other costs” include interest paid on the market 
debt and interest related to the superannuation accounts (see 
Appendix A). Combining them in the Estimates obscures the fact that 
a third of the projected interest charges do not involve payment in 
cash. Fluctuations in projected interest charges due to variations in 
interest related to the superannuation accounts or interest charges on 
market debt cannot be easily distinguished. In our 1991 Report, 
Chapter 8—Debt Management and Employee Pensions, we observed 
that the reporting of interest charges was confusing because the 
interest expense relating to pension accounts was charged to the 
Department of Finance Canada as part of total public interest charges. 
The Department responded that it would consider the merit of 
disclosing further information. Forecasted expenditures mixing cash 
with transactions not involving cash are not limited to the “interests 
and other costs.” They are commonly used in the Estimates. This 
presentation is unclear and can confuse parliamentarians and the 
public about what is included in the Estimates. 


3.62 Recommendation. Given the magnitude of charges related to 
the superannuation accounts, the Treasury Board of Canada 
Secretariat and the Department of Finance Canada should improve 
the clarity of information on interest charges by separating, in the 
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Public Sector Pension Investment Board—An 
investment board created in 2000 and 
responsible for managing the assets held in the 
public sector pension funds for post-2000 
contributions to pension plans. 
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Estimates, the projected interest charges for the market debt from 
other interest charges in order to improve transparency. 


The Department’s response. Agreed. The Department of Finance 
Canada will work with the Treasury Board of Canada Secretariat to 
present a breakdown of interest and other costs in the Estimates. 


The Secretariat’s response. Agreed. The Department of Finance 
Canada will produce for the Estimates a breakdown of interest and 
other costs, starting with the 2012—13 Estimates, which the Treasury 
Board of Canada Secretariat will publish. 


3.63 Information on public sector pension investments. Changes in 
market conditions could cause public sector pension plan assets, net 
liabilities, and related interest charges to fluctuate. We found that the 
Department, with the exception of the annual report, does not have 
timely access to quarterly information about actual returns on public 
sector pension investments to assess the impact that unforeseen 
fluctuations could have on budget surpluses or deficits. 


3.64 Interest expenses credited to superannuation accounts (pre-2000 
contributions) are calculated using a formula that sets the interest rate. 
The interest expenses associated with accrued pension benefits (for 
both pre and post-2000 pension obligations) are determined based on 
the results of the actuarial valuations performed for accounting 
purposes and the application of generally accepted accounting 
principles for employee future benefit accounting. The methods used 
to calculate these interest expenses offset year-over-year volatility, 
limiting the potential effects on annual interest charges in the fiscal 
year where the market conditions changed. Interest charges associated 
with these liabilities are subtracted from the return on assets managed 
by the Public Sector Pension Investment Board (PSP Investments) 
(see Appendix A). Even though the return on assets is also smoothed, 
as the size of assets of the public sector pension funds increases over 
time, significant changes in investments performance could have a 
large cumulative impact on the budgetary deficit or surplus. According 
to PSP Investments’ 2011 Annual Report, consolidated net assets are 
likely to reach some $300 billion by 2030. 


3.65 During the period under review, growth in assets under PSP 
Investments and fluctuations on the return on total assets led to 
annual variations of $300 million to $700 million in public interest 
charges. Such variations need to be actively monitored, especially in 
the context of increasing assets under management and larger market 
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volatility. If not monitored, the government’s ability to project its 
budgetary deficit or surplus and meet its target could be jeopardized. 


3.66 The Department is not provided with a quarterly update report 
on PSP Investments’ performance under the Public Sector Pension 
Investment Board Act. In the future, as the size of assets increases, this 
financial information would be helpful for the Department to forecast 
the impact that variations in returns on pension funds assets could 
have on future deficits or surpluses. 


3.67 Recommendation. The Department of Finance Canada should 
seek interim information regarding the actual return of public sector 
pension plan investments in order to properly assess their impact on 
budgetary deficits or surpluses, as their potential impact on interest 
charges will grow with the size of assets. 


The Department’s response. Agreed. The Department of Finance 
Canada will work with the Treasury Board of Canada Secretariat to 
establish a process for the sharing of interim financial information 
regarding public sector pension plan investments. 


The Department would like to note, however, that it already has access 
to sufficient information to properly assess the impact of investment 
returns on the budgetary balance. As a result, the added value of this 
interim financial data is considered minimal. 


3.68 Information on public sector pension plan liabilities. Pension 
accounting and actuarial valuation of pension liabilities are complex 
subject areas that can be difficult for people to understand. Thus, it is 
necessary for management to provide clear information. We found that 
there is no clear information available to help Canadians understand 
pension liabilities and how these liabilities and related interest charges 
are calculated. We also found that the government reporting in this 
area could be improved by presenting the information in a more easily 
understandable format. 


3.69 No single organization is responsible for monitoring or reporting 
information on the public sector pension plan liabilities. To gather 
information on the public sector pension plan assets, liabilities, and 
related interest payments, users must refer to nine separate reports that 
contain complex and fragmented information: 


¢ Public Accounts of Canada, 
¢ the PSP Investments Annual Report, 


* the Report on the Public Service Pension Plan, 
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¢ the Actuarial Report on the Pension Plan of the Public Service 
of Canada, 


¢ the Annual Report: Canadian Forces Pension Plan, 


¢ the Actuarial Report on the Pension Plan of the Canadian 
Forces—Regular Force, 


¢ the Actuarial Report on the Pension Plan of the Canadian 
Forces—Reserve Force, 


¢ the Annual Report: Royal Canadian Mounted Police Pension 
Plan, and 


¢ the Actuarial Report on the Pension Plan for the Royal Canadian 
Mounted Police. 


Each of these reports has its own purpose and audience. The Public 
Accounts of Canada Volume I includes notes to the Financial 
Statements of the Government of Canada in section 2 and section 6 
on the interest-bearing debt. The annual reports on pension plans 
provide details on operations of the individual plans for the latest fiscal 
year. The actuarial reports, published for each plan every three years, 
report on the state of the different components of each pension plan to 
help the responsible minister make informed decisions about financing 
the government’s pension benefit obligation. The PSP Investments 
Annual Report provides information on the pension funds’ investment 
return of the assets under management, among other things. 


3.70 In Canada, the government has not consolidated information on 
the public sector pension plans in an easy-to-read document showing 
how these liabilities affect the interest-bearing debt. While this 
information is captured in part in the Public Accounts of Canada and 
other documents, parliamentarians and the public do not have access 
to information in an easy-to-read format that would help them 
understand how these liabilities affect the government’s overall fiscal 
strength. 


3.71 In our view, information on the public sector pension plan 
liabilities could be reported in a better way. It could include a complete 
description of the methodology, the assumptions, and the discount 
rates used to assess the liabilities as well as the interest charges related 
to public sector pension plans. This information could also include the 
projected fiscal impact of these liabilities. Such reporting would allow 
parliamentarians and Canadians to understand the financial 
implications of the public sector pension plan liabilities. 
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3.72 Recommendation. The Treasury Board of Canada Secretariat 
and the Department of Finance Canada should report, in a 
consolidated manner, clear and understandable information on the 
public sector pension plan liabilities (including supporting 
methodology and assumptions) and should explain their impact on the 
government’s finances. 


The Department’s response. Agreed. The Public Accounts of Canada 
is intended to provide clear and understandable summary information 
on the government’s public sector pension plan liabilities and their 
impact on the government's financial results. The Department of 
Finance Canada will work with the Treasury Board of Canada 
Secretariat to look at ways of improving the presentation of public 
sector pension plan information in the Public Accounts. 


The Secretariat’s response. Agreed. The Public Accounts of Canada 
is intended to provide clear and understandable summary information 
on the government’s public sector pension plan liabilities and their 
impact on the government’s financial results. The Treasury Board of 
Canada Secretariat will work with the Department of Finance Canada 
to look at ways of improving the presentation of public sector pension 
plan information in the Public Accounts, starting with the 2013 Public 
Accounts. 


Conclusion 


3.73 Public debt and the associated interest charges consume a large 
amount of financial resources. They affect the government’s fiscal 
strength, limit policy choices, and influence what we can afford as a 
nation. In this context, the interest-bearing debt needs to be managed 
and reported on properly. Clearer information allows for better policy 
debate, including debates on how to keep public finances sustainable. 


3.74 We found that the Department of Finance Canada has put in 
place a sound decision-making system to support and develop effective 
market debt strategies. The Department has introduced a debt strategy 
model that projects budgetary deficits or surpluses, associated interest 
charges, and budgetary risk. The model has allowed debt managers to 
design debt management strategies from a broader perspective on fiscal 
planning. The model and the supporting quantitative analyses are 
major advances that help debt managers identify sets of desirable debt 
structures. Also, the Department uses judgment and consults with 
market participants to support the government’s debt management 
strategy. 
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3.75 We found that the overall performance of the debt management 
strategy needs to be better monitored and reported on to inform 
management decisions. The Department uses indicators to monitor 
the performance of its debt management strategy against the 
government’s objective of maintaining a well-functioning market for 
Government of Canada securities. The strategy is achieving this 
objective. By using more appropriate indicators, the Department can 
now better monitor the performance of this strategy against its other 
objective to raise low-cost and stable funding for the government by 
balancing the cost and risks associated with the debt structure. It has 
yet to demonstrate that the strategy is achieving this objective. 


3.76 The Department of Finance Canada publishes clear information 
on market debt. However, it should publish more details, such as how 
it justifies the debt management strategy or the overall performance of 
the program. 


3.77 We found that the Department has a sound risk management 
framework to measure and monitor risks and changes in financial 
requirements that could affect the debt management strategy. The 
Department has adequate processes to update and adapt the strategy. 
However, the Department needs to expand its marketing tools and 
promotional activities in order to reach a broader base of investors. 


3.78 Relative to other countries, Canada is a leader in reporting 
public sector pension plan liabilities in its financial statements. 
However, the information provided to parliamentarians and Canadians 
on the public sector pension plans is not easy to understand and is 
dispersed. Because funding the interest-bearing debt has an impact on 
the government's surplus or the deficit, complete and clear information 
on this debt should be reported. Clearer information would allow for 
better policy debates and choices. The reporting of forecasted interest 
charges in the Estimates needs to be improved in order to be more 
transparent. Finally, we noted that the Department does not have 
timely access to the interim information on the return of public sector 
pension investments in order to assess how unforeseen fluctuations 
could affect budgetary surpluses or deficits. 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these 
standards as the minimum requirement for our audits, we also draw upon the standards and practices of 
other disciplines. 


Objectives 


The audit examined whether the Department of Finance Canada and the Treasury Board of Canada 
Secretariat, consistent with their respective responsibilities, effectively managed the interest-bearing debt 
of the Government of Canada. The audit had the following sub- objectives: 


+ to determine whether the Department implemented a sound decision-making system to support and 
develop effective market debt strategies, 


* to determine whether the debt management strategy responded to emerging risks and changing 
funding requirements, 


* to determine whether the Department assessed the performance of the debt management strategy by 
establishing objectives and monitoring performance as well as used performance results to support debt 
management decisions, and 


* to determine whether the Department and the Treasury Board of Canada Secretariat reported 
comprehensive and clear information on the budgetary impact of the public sector pension plans. 


Scope and approach 


The audit included the Department of Finance Canada and the Treasury Board of Canada Secretariat. At 
the Department of Finance Canada, we looked at the Financial Markets Division and the Fiscal Policy 
Division. At the Treasury Board of Canada Secretariat, we discussed issues with officials from the Pensions 
and Benefits Sector, the Expenditure Management Sector, and the Office of the Comptroller General of 
Canada. We also interviewed officials from the Bank of Canada and from the Office of the Chief Actuary. 


The audit focused on the process used by the Department of Finance for selecting market debt strategies, 
for measuring performance, for managing risks that could have an impact on debt strategies, and for 
reporting information on the public debt. The audit also examined the information provided by the 
Treasury Board of Canada Secretariat and the Department of Finance Canada on the interest-bearing 
debt. More specifically, we focused on how interest charges are reported in the Estimates and how public 
sector pension plan liabilities are reported in order to inform parliamentarians and Canadians. 


We reviewed various documents, including reports, minutes of meetings, analyses, research papers, as well 
as foreign reports, including guidance issued by international organizations. We also reviewed literature 
related to the interest-bearing debt. In addition, we interviewed debt managers in selected countries and 
consulted with experts in the field. We examined the market debt management activities carried out by 
the Department of Finance Canada from 1 April 2007 to 31 March 2011. We did not question the 
appropriateness of the debt management strategy or funding decisions. We examined practices for 
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reporting information on the interest-bearing debt at the Department of Finance Canada and the Treasury 
Board of Canada Secretariat that were in place as of 31 March 2011. 


Criteria 


The Department of Finance Canada performs analyses using ¢ Guidelines for Public Debt Management, International 

quantitative tools (including modelling analyses) as well as Monetary Fund and World Bank 

qualitative information to establish trade-offs between costs and 
risks as well as to maintain liquid and well-functioning markets 

for Government of Canada debt. 


¢ Stockholm Principles, International Monetary Fund 


Developing a Medium-Term Debt Management Strategy 
(MTDS): The Analytical Tool User Guide, International 
Monetary Fund and World Bank 


The Department of Finance Canada ensures that the robustness | * Departmental response to the Risk Management Report 
of the quantitative analyses is assessed and challenges the prepared for the Departmental Finance Canada 
projected results of funding scenarios. 


¢ Departmental response to the Evaluation of the Government's 
Decision to Target a Higher Fixed Rate Debt Structure, 
Department of Finance Canada 


Developing a Medium-Term Debt Management Strategy 
(MTDS): The Analytical Tool User Guide, International 
Monetary Fund and World Bank 


The Department of Finance Canada has a sound risk Guidelines for Public Debt Management, International 
management governance framework for measuring and Monetary Fund and World Bank 

monitoring risks as well as changes in financial requirements that 
could have an impact on the debt strategy. 


Stockholm Principles, International Monetary Fund 


e Advances in Risk Management of Government Debt, 
Organisation for Economic Co-operation and Development 


The Department of Finance Canada has adequate processes to * Debt Management Strategy 2011-12, Department of Finance 
update and adapt the financial strategy when’ new risks emerge Canada 
or when financial requirements change. 


Guidelines for Public Debt Management, International 
Monetary Fund and World Bank 


Stockholm Principles, International Monetary Fund 


Enterprise Risk Management—Integrated Framework, 
Committee of Sponsoring Organizations of the Treadway 
Commission 


Principles for Stable Capital Flows and Fair Debt 
Restructuring, Institute of International Finance 
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The Department of Finance Canada establishes performance Guidelines for Public Debt Management, International 
measurement mechanisms and targets to monitor the overall Monetary Fund and World Bank 
performance of the debt management strategy using metrics 


against the objectives set for the program. ¢ Debt Management Performance Assessment (DeMPA) Tool, 


World Bank 


e Advances in Risk Management and Government Debt, 
Organisation for Economic Co-operation and Development 


The Department of Finance Canada considers past performance Guidelines for Public Debt Management, International 
results to update its debt strategy. Monetary Fund and World Bank 


¢ Debt Management Performance Assessment (DeMPA) Tool, 
World Bank 


e Advances in Risk Management and Government Debt, 
Organisation for Economic Co-operation and Development 


The Department of Finance Canada and the Treasury Board of e¢ CICA Handbook, Canadian Institute of Chartered Accountants 
Canada Secretariat publish accurate, complete, and clear 
information on the budgetary impact of the public sector pension 


plans. ¢ Code of Good Practices on Fiscal Transparency, International 
Monetary Fund 


¢ PSAB Handbook, Canadian Institute of Chartered Accountants 


¢ Manual on Fiscal Transparency, International Monetary Fund 


Best Practices for Budget Transparency, Organisation for 
Economic Co-operation and Development 


Management reviewed and accepted the suitability of the criteria used in the audit. 


Period covered by the audit 

The audit covered the period from 1 April 2007 to 31 March 2011. Audit work for this chapter was 
substantially completed on 31 October 2011. 

Audit team 


Assistant Auditor General: Nancy Cheng 
Principal: Richard Domingue 
Director: Philippe Le Goff 


Johnathon Cziffra 


Rose Pelletier 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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Appendix A Recent events that have had an impact on Canada’s interest-bearing debt 


Policy changes for public sector pension plans. In 1999, Parliament enacted the Public Sector Pension 
Investment Board Act, amending the Public Service Superannuation Act, the Canadian Forces Superannuation 
Act, and the Royal Canadian Mounted Police Superannuation Act. These acts are collectively known as the 
superannuation acts. The amendments came into force on 1 April 2000, and new funds were created for 
each plan (the pension funds) to receive all employee and government pension contributions made after 
that date. Hence, contributions and benefit payments for pensionable service accrued after 31 March 2000 
are recorded in the pension fund accounts. An investment board—the Public Sector Pension Investment 
Board, or PSP Investments—was set up to manage the assets in the pension funds. Transactions such as 
contributions, benefits paid, and transfers that relate to service provided before 1 April 2000 are recorded 
in the superannuation accounts, which are credited with interest as though invested in a portfolio of 
Government of Canada long-term bonds (20-year and more) held to maturity. The average interest rate 
credited to the accounts was 6.7 percent in the 2009-10 fiscal year. In essence, the superannuation 
accounts are legislated ledgers and do not hold assets. 


New borrowing authority process. Prior to 2007, the Financial Administration Act gave the government 
standing authority to refinance its market debt, while specific authority was to be granted by Parliament to 
undertake additional borrowing beyond an existing $4 billion of non-lapsing borrowing authority. In the 
Budget Implementation Act, 2007, the $4 billion limit was replaced by establishing borrowing authority 
under the Governor in Council that authorizes the Minister of Finance to borrow money. Annual 
borrowing limits are now approved by the Governor in Council. Parliament no longer has to approve the 
borrowing limits of the government. For 2011-12, the aggregate borrowing limit approved by the 
Governor in Council is $300 billion. 


Consolidation of some Crown Corporations borrowings. In 2008, the government consolidated the 
borrowings of three financial Crown corporations—the Business Development Bank of Canada, the 
Canada Mortgage and Housing Corporation, and Farm Credit Canada—into the federal debt program. As 
federal debt was rapidly declining, increasing the issuing of Government of Canada securities (a result of 
consolidated borrowing) was seen as a way to make the Government of Canada bond market more liquid. 
This approach was also aimed at reducing borrowing costs of the three corporations. According to the 
Department of Finance Canada, the consolidated borrowings of these Crown corporations have since 
grown to account for $34 billion of federal market debt. This Crown borrowing activity does not affect the 


accumulated deficit (federal debt), since increased federal borrowing is matched by assets in the form of 
loans to the Crown corporations. 


Financial turmoil and stimulus measures. During the second half of the 2007-08 fiscal year, financial 
markets were turbulent because of the sharp decline in the United States housing market and concerns 
about the creditworthiness of financial institutions. In January 2009, the federal government announced a 
series of measures to stimulate the economy. The Economic Action Plan totalled more than $46 billion in 
new federal initiatives. The Extraordinary Financing Framework (EFF) initiative was created to make 
credit more available and to respond to gaps in credit markets so that credit restrictions would not deepen 
Canada’s economic downturn. For example, the government provided more than $69 billion in EFF 
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support to maintain the availability of long-term credit by purchasing mortgage-related securities. Because 
of the impact of the financial turmoil on revenues and expenditures, the government posted budgetary 


deficits of $5.8 billion in 2008-09, $55.6 billion in 2009-10, and $33.4 in 2010-11. 
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Appendix B List of recommendations 


The following is a list of recommendations found in Chapter 3. The number in front of the 
recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses 
indicate the paragraphs where the topic is discussed. 


3.51 The Department of Finance The Department’s response. Agreed. The Department of 
Canada should provide better financial | Finance Canada’s website contains a wide range of materials on 
information to market participants and Canada’s borrowing programs, debt strategy, and economic and 


Management of emerging risks 


expand its communication and fiscal performance. The Bank of Canada’s website also contains 
marketing tools, as well as its details on Canada’s debt auctions. That said, the Department 
promotional activities in order toreach and the Bank plan to expand the range of economic, financial, 
a broader base of investors. and borrowing information available on their websites, provide 
(3.48-3.50) ' better links between the two sites, and improve the organization 


of this information to make it more easily accessible. The 
Department recognizes the importance of maintaining and 
building on Canada’s strong reputation in global capital markets 
and will take steps to increase its engagement with domestic and 
international investors as opportunities present themselves. 


In terms of expanding promotional activities, the Department 
would note that the investor base for Government of Canada 
securities has grown in recent years, that debt auctions have 
been well covered, and that Canada’s global foreign-currency 
bond issues have met with very strong demand from 
international investors. Devoting significantly more resources to 
investor relations and associated promotional activities would 
not provide value for money. 
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Reporting information on the interest-bearing debt 


3.57 The Department of Finance 
Canada should improve its reports on 
the market debt by including the 
analysis that supports the debt 
management strategy and information 
on the overall performance results 
against planned outcomes. These 
improvements will help better inform 
parliamentarians, Canadians, and 
investors on the actual results of the 
debt management strategy. 


(3.54—3.56) 


3.62 Given the magnitude of charges 
related to the superannuation accounts, 
the Treasury Board of Canada 
Secretariat and the Department of 
Finance Canada should improve the 
clarity of information on interest 
charges by separating, in the Estimates, 
the projected interest charges for the 
market debt from other interest charges 
in order to improve transparency. 


(3.58-3.61) 


3.67 The Department of Finance 
Canada should seek interim 
information regarding the actual return 
of public sector pension plan 
investments in order to properly assess 
their impact on budgetary deficits or 
surpluses, as their potential impact on 
interest charges will grow with the size 
of assets. (3.63—3.66) 


The Department’s response. Agreed. Where feasible, the 
Department of Finance Canada will aim to improve the 
information content of our reports. 


We would note, however, that compared to its peers, Canada is 
already among the most informative in terms of its debt 
management reporting. The Debt Management Strategy and 
Debt Management Report are among the most transparent 
documents of the G20 countries with respect to describing the 
government's debt strategy as well as providing performance 
outcomes. New metrics have been added to recent reports. The 
Department also publishes external evaluations of debt 
management programs and operations conducted under the 
Treasury Evaluation Program. Additionally, a number of papers 
on Canada’s stochastic debt strategy model are available on the 
Bank of Canada’s website. 


The Department’s response. Agreed. The Department of 
Finance Canada will work with the Treasury Board of Canada 
Secretariat to present a breakdown of interest and other costs in 
the Estimates. 


The Secretariat’s response. Agreed. The Department of 
Finance Canada will produce for the Estimates a breakdown of 
interest and other costs, starting with the 2012—13 Estimates, 
which the Treasury Board of Canada Secretariat will publish. 


The Department’s response. Agreed. The Department of 
Finance Canada will work with the Treasury Board of Canada 
Secretariat to establish a process for the sharing of interim 
financial information regarding public sector pension plan 
investments. 


The Department would like to note, however, that it already has 
access to sufficient information to properly assess the impact of 
investment returns on the budgetary balance. As a result, the 
added value of this interim financial data is considered minimal. 
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3.72 The Treasury Board of Canada The Department’s response. Agreed. The Public Accounts of 


Secretariat and the Department of Canada is intended to provide clear and understandable 
Finance Canada should report, in a summary information on the government’s public sector pension 
consolidated manner, clear and plan liabilities and their impact on the government’s financial 
understandable information on the results. The Department of Finance Canada will work with the 
public sector pension plan liabilities Treasury Board of Canada Secretariat to look at ways of 
(including supporting methodology and —_ improving the presentation of public sector pension plan 
assumptions) and should explain their information in the Public Accounts. 


impact on the government’s finances. 


(3.68-3.71) The Secretariat’s response. Agreed. The Public Accounts of 


Canada is intended to provide clear and understandable 
summary information on the government’s public sector pension 
plan liabilities and their impact on the government's financial 
results. The Treasury Board of Canada Secretariat will work with 
the Department of Finance Canada to look at ways of improving 
the presentation of public sector pension plan information in the 
Public Accounts, starting with the 2013 Public Accounts. 
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CHAPTER 4 
Non-Filers and Non-Registrants—Canada Revenue Agency 


Performance audit reports 


This report presents the results of a performance audit conducted by the Office of 
the Auditor General of Canada under the authority of the Auditor General Act. 


A performance audit is an independent, objective, and systematic assessment 

of how well government is managing its activities, responsibilities, and resources. 
Audit topics are selected based on their significance. While the Office may 
comment on policy implementation in a performance audit, it does not comment 
on the merits of a policy. 


Performance audits are planned, performed, and reported in accordance with 
professional auditing standards and Office policies. They are conducted by 
qualified auditors who 


* establish audit objectives and criteria for the assessment of performance; 
¢ gather the evidence necessary to assess performance against the criteria; 
* report both positive and negative findings; 

* conclude against the established audit objectives; and 


* make recommendations for improvement when there are significant 
differences between criteria and assessed performance. 


Performance audits contribute to a public service that is ethical and effective 
and a government that is accountable to Parliament and Canadians. 


Table of Contents 


Main Points 


Introduction 


Focus of the audit 


Observations and Recommendations 


Identifying non-filers and non-registrants 

The Agency has validated its process for selecting files but not for rejecting files 
The Agency needs to validate its process for registering potential non-registrants 
File selection for manual pursuit needs improvement 

The Agency does not pursue all non-filing corporations 

Identification projects find non-filers for whom the Agency lacks information 
Tax services offices rarely follow the new project approval process 

Planning work 

The annual planning process focuses on production targets 

The Agency considers the cost of achieving filing compliance when setting objectives 
Measuring results and making improvements 


Performance indicators measure only routine program activities 
The Agency has not integrated research results into planning to improve the program 


Conclusion 
About the Audit 


Appendix 


List of recommendations 
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What we examined 


Why it’s important 


Non-Filers and Non-Registrants— 
Canada Revenue Agency 


Main Points 


The Canada Revenue Agency works to ensure that Canadians pay 
their required share of taxes and that the revenue base is protected. 
It is responsible for administering the Income Tax Act, which specifies 
when taxpayers are required to file a return. A non-filer is an 
individual, a corporation, or a trust who fails to file a tax return as 
required by legislation. 


Under the Excise Tax Act, businesses that meet certain criteria are 
required to register for the goods and services tax/harmonized sales tax 
(GST/HST). Non-registrants are businesses that fail to comply with 
this requirement. 


The Agency, through its Non-Filer/Non-Registrant (NF/NR) program, 
works to encourage individuals, corporations, and trusts to comply 
with the filing requirement and, in the case of businesses, with the 
GST/HST registration requirements. In the 2010-11 fiscal year, the 
NEF/NR program’s salary budget was $39 million of the Agency’s total 
budget of $4.5 billion, and it employed 700 of the Agency’s 
approximately 39,000 employees. This relatively small program area 
generated $2.8 billion of additional taxes, interest, and penalties 
assessed in each of the 2009-10 and 2010-11 fiscal years. 


We looked at what the Agency has done to address non-compliance 
with filing and registration requirements. The audit focused on how 
the Agency identifies non-filers and non-registrants and how it plans, 
monitors, and reports on its actions to improve compliance. 


Audit work for this chapter was substantially completed on 
5 January 2012. Further details on the conduct of the audit are in 
About the Audit at the end of the chapter. 


Income tax is the single most important source of government 
revenue. For the income tax system to operate fairly and effectively, 

it is important that all individuals and corporations file returns when 
they are required by law to do so, and that they pay the taxes they owe. 
Not doing so reduces the amount of money available for important 
government programs such as health care, education, and the 
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What we found 


environment. Similarly, businesses that are required to register for GST 
or HST but do not, may have a negative impact on government 


revenues. The Agency has identified non-filers and non-registrants as 


high priority issues to address, as part of its efforts to combat non- 


compliance, including the underground economy. 


- Because the Agency does not have the resources to pursue all non- 


filers, it has developed a risk scoring model to identify those it will 
pursue. Agency analysis indicated that the files it chooses to pursue 
result in returns filed and taxes assessed. However, it has not tested 
its screening to determine whether the files it chooses not to pursue 
from the initial 2.5 to 3 million files identified by the matching 
process should in fact be pursued. In addition, two thirds of the files 
initially selected for pursuit in the field are later determined to have 
low potential, and work on them is discontinued. In other words, the 
Agency does not know if its risk-scoring process is as effective as it 


could be. 


The Agency uses identification projects to find taxpayers who may 
be participating in the underground economy. The majority of 
projects completed were meeting or exceeding their targets for 
number of tax returns filed and amounts assessed. 


The Agency’s planning process tends to repeat actions from one year 
to the next to identify non-filers and non-registrants. While it 
considers the cost of its processes to pursue these files when planning 
work, the Agency is not taking advantage of its research findings in 
its work plans in order to improve its results in achieving compliance. 


Current performance indicators focus on routine program 
activities—for example, tax dollars and the number of taxpayers 
assessed—rather than on the longer term impact of the program, in 
particular the Agency’s success at getting non-compliant taxpayers 
to change their behaviour and file their returns. Various audits and 
program evaluations over the years have recommended action to 
develop additional performance measures and risk management of 
the Non-Filer/Non-Registrant (NF/NR) program and to improve its 
use of internal and external information. The Agency has made 
limited progress in implementing many of these recommendations. 


The Agency has responded. The Agency agrees with all of the 
recommendations. Its detailed responses follow the recommendations 
throughout the chapter. 
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any person who resides in Canada, whether or 
not they are liable to pay tax. For tax purposes, 
a person may be an individual, a corporation, 
or a trust. 
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Introduction 


41 The mission of the Canada Revenue Agency is to administer tax, 
benefits, and related programs and to ensure taxpayer compliance, on 
behalf of governments across Canada. As Canada’s tax administrator, 
in addition to protecting Canada’s revenue base, the Agency’s primary 
goal is compliance—ensuring that taxpayers meet their obligations. 
Under the Canada Revenue Agency Act, the Agency is responsible for 
enforcing the Income Tax Act and the Excise Tax Act. 


42 As part of fulfilling that responsibility, the Agency has created 
the Non-Filer/Non-Registrant (NF/NR) program. The program’s 
mandate is to encourage individuals, corporations, and trusts to 
comply with their requirements under the Income Tax Act to file tax 
returns as well as to encourage businesses to register for the goods and 
services tax/harmonized sales tax (GST/HST). The NF/NR program’s 


stated objective is 


to develop fair, responsible and effective strategies to achieve 
high levels of filing and registration compliance commensurate 
with the level of funding while generating a high return on 
investment (fiscal impact and production [returns obtained]). 


4.3 The Income Tax Act identifies situations that require individuals 
to file returns, which they must do to receive income tax refunds. 
Since many individuals are employees, and income tax is deducted at 
source from their pay, many receive refunds once they have filed their 
returns. Therefore, filing is in their best interest. 


4.4 Filing an income tax return is the first step in the compliance 
continuum. The Agency assesses taxes once a taxpayer has filed a 
return; if necessary, it may also assess taxes even if a taxpayer has not 
filed a return. Once the return is filed, if taxes are owed, a different 
compliance risk may arise—that of whether a taxpayer can or will pay 
the amount owed. Taxpayers may not file their returns for a variety of 
reasons. Some forget, some procrastinate, and others do not file their 
returns or do not report their income in an effort to avoid tax. The 
Agency provides tools to taxpayers to help them comply and, when 
necessary, intervenes with a variety of actions. 


45 An individual taxpayer is required to file a return when one of 
the following criteria is met: 


¢ Tax is payable. 


¢ The Agency requests or demands a return. 
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* The taxpayer has elected to split pension income with a spouse or 
common-law partner. 


¢ The taxpayer wishes to receive the Working Income Tax Benefit. 


¢ The taxpayer has disposed of capital property or realized a taxable 
capital gain. 


¢ The federal government has overpaid Old Age Security or 
Employment Insurance benefits. 


* The taxpayer owes money to a Registered Retirement Savings 
Plan (RRSP) because of the Home Buyers’ Plan or the Lifelong 
Learning Plan. 


* The taxpayer is required to contribute to the Canada Pension Plan. 


Individuals, corporations, or trusts who do not file a tax return as 
required are called “non-filers.” 


46 The Agency administers benefits on behalf of other government 
departments as well as other levels of government. Often, these 
benefits are based on information in the income tax return. Unless 
individuals file a tax return, they cannot receive the benefits for which 
they may be eligible. The Agency aims to facilitate access to benefits, 
such as child benefits and the GST/HST credit, through its service 
delivery programs. 


4.7 In addition, under the Excise Tax Act, businesses must register for 
the GST or HST if they provide goods or services equivalent 

to $30,000 or more each year. Charities must register after 

earning $250,000 in revenue; for public service bodies, such as schools, 
universities, and hospitals, the threshold is $50,000. Taxi drivers and 
certain non-resident businesses must register regardless of revenue. 
Businesses that should register but do not are called “non-registrants.” 


48 Approximately 700 of the Agency’s 39,000 employees work in 
the NF/NR program. In the 2010-11 fiscal year, the program’s budget 
was $39 million of the Agency’s total budget of $4.5 billion. Compared 
to the other ways the Agency deals with compliance issues, such as 
audits and income tax collections, the program is relatively small. 
Despite its small size, the NF/NR program generated $2.8 billion of 
additional taxes, interest, and penalties in each of the 2009-10 and 
2010-11 fiscal years. 
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Underground economy—lypically involves 
commercial activity that is unreported for tax 
purposes. Underground economic activity is 
particularly prevalent in industry sectors where 
cash transactions are common, such as 
hospitality, automotive repairs, and construction, 
including home renovation. 


Source: Canada Revenue Agency 


Organisation for Economic Co-operation and 
Development (OECD)—An organization 
consisting of member countries. It conducts 
research and creates agreements, standards, 
and recommendations to end poverty through 
economic growth and financial stability. One of 
the OECD’s focus areas is taxation. 
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49 The Agency has identified the pursuit of non-filers and non- 
registrants as a high priority in its strategy to combat non-compliance, 
including the underground economy. The Agency’s strategy 
incorporates 


* research, to understand non-compliance; 


* operations, to correct non-compliance through effectiveness and 
efficiency; 


* communications, to promote voluntary compliance; 
¢ legislative changes, when necessary; and 


* partnerships with the Organisation for Economic Co-operation 


and Development (OECD). 


410 The Agency recently reviewed the delivery model for its NF/NR 
program. The review looked at operational processes, workload 
management, and resource use to identify improvements that would 
result in better returns on investments for the program. In 

March 2011, the Agency summarized its findings, and it is currently 
reviewing the results. 


Focus of the audit 


4.11 The objective of the audit was to determine whether the Canada 
Revenue Agency adequately addressed filing non-compliance (non- 
filers) and registration non-compliance (non-registrants). 


4.12 We looked at how the Agency planned suitable methods to 
address both types of non-compliance, whether these plans 
incorporated past experience and research, and whether they took into 
account the cost of using these methods. We also looked at how the 
Agency validated its selection decisions and how well its projects 
addressed non-filing risks. Finally, we analyzed the way the Agency 
monitored and reported its actions toward non-filers and non- 
registrants. In all of these areas, we considered past audit and 
evaluation results, both internal and external, and recommendations 
the Agency had previously accepted. 


4.13 We did not focus on the issue of whether taxpayers are paying 
the taxes that had been assessed. 


4.14 More details about the audit objectives, scope, approach, and 
criteria are in About the Audit at the end of this chapter. 
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Identifying non-filers 
and non-registrants 


Demand to file—the formal notification of a 
taxpayer's requirement to file an income tax 
return—based on subsection 150(2) of the 
Income Tax Act. 


Observations and Recommendations 


415 Most taxpayers in Canada file their tax returns on time. Others 
file late before Agency intervention. The Agency will assess interest 
and penalties for late-filed returns. If a tax return has not been filed, 
the Agency will start the non-filer process. 


4.16 There are two types of non-filers: known and unknown. 


Known nopr-filers are those that the Agency can identify. It does this 


by matching information slips (for example, T4 slips reporting 


employment income and T5 slips reporting investment income) filed 


by employers or other entities to tax returns filed by taxpayers. 


4.17 Unknown nonr-filers are those the Agency cannot identify 


through information slips or other documents. They are often self- 
employed or operate within the underground economy. Because there 
is little or no automated information about unknown non-filers and 
they are harder to find, they are at higher risk of going undetected than 
known nopr-filers. 


4.18 ‘To start the identification process, Agency computer systems 
match information slips to personal tax returns to determine if a 


taxpayer has filed a return and if the amount reported is correct. 
Information slips that do not have matching tax returns identify 


potential non-filers. Each year, this process identifies 2.5 to 3 million 
individuals who are potential non-filers. Many are not determined to 
be non-filers, since they may not be obliged to file a return, or they may 
file their returns late. Another automated process identifies corporate 


non-filers by matching their corporate business numbers to corporate 
income tax returns. 


4.19 Once this initial matching process has identified potential non- 


filers, the Agency applies a risk-scoring process to select which 
taxpayers to pursue. The initial process for pursuing them is automated 


(Exhibit 4.1). 


* The Agency sends letters, reminding them to file their 
outstanding returns. 


* If the Agency does not receive a return within 40 days, it may send 
a second “request to file” letter or it may send a demand to file. 


* Finally, ifit has not received a return, the Agency may transfer the 
file to a tax centre or a tax services office where a staff member 
will begin a manual follow-up process. In some cases, the process 
ends with the prosecution of a taxpayer, resulting in a fine or jail 
term or both. 
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The Agency has validated its process for selecting files but not for rejecting files 


4.20 We looked at whether the Agency validates (tests the validity of) 
its screening process for identifying non-filers, and whether it knows if 
the process was appropriate in identifying files to pursue. It is important 
to validate this process, because it could include (“screen in”) individuals 
who owe little or no tax or who are due a refund (low potential), and 
exclude (“screen out”) individuals who owe high amounts of taxes (high 
potential). For 2010 and 2009, we examined whether the Agency 
validated its screening process to confirm that it was excluding low 
potential files and including high potential files. 


Exhibit 4.1 Process for pursuing individual non-filers, using the 2009 tax year as an example 


Returns 
may be filed 
at any time 


Filing deadline for 2009 tax year—30 April 2010 


2.5 million potential non-filers identified T1 matching 


Escalating 
enforcement 
action 


OR 


¢ Prosecutions 


Potential non-filers selected for action 


“Request to File” letters sent 


Second “Request to File” letter 


“Demand to File” letter 


¢ Staff may be assigned to follow up 
¢ Tax payable may be assessed— 
Income Tax Act, subsection 152(7) 


Positive result 


Return filed 
Tax assessed: $1.3 billion* 


* Net tax owed: $1.5 billion, 57% of returns 
+ Net tax refunded: $215 million, 36% of returns 
No tax owed or refunded: 7% of returns 


The files considered 
to be low risk are not 
pursued in the current year 
and may have 
no obligation to file 
a tax return 


Risk assessment 
(T1 risk scoring) 


Automated 
strategies start 


Strategies after 
40 days elapse 


Strategies after 
45 days elapse 


Positive result 


Some of those not selected 
for pursuit will file a return 


Negative result 


Return not filed 
File discontinued 


*Total taxes assessed does not match the annual report, because this is the T1 process for individuals and does not include corporations, trusts, or GST/HST. 
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4.21 According to guidance from the Organisation for Economic 
Co-operation and Development (OECD) on how to evaluate the 
effectiveness of an organization’s strategies to encourage compliance, it 
is important to validate the process that the organization uses to select 
cases. This process is described in Exhibit 4.2. A flawed selection 
process could result in high costs and poor results, whereas sound 
selection processes result in low costs and positive results. 


Exhibit 4.2 The Compliance Risk Management Process 


Operating context 


Assess and prioritise risks 


Monitor performance 
against plan 


Evaluate compliance 
outcomes 


* Registration 


Analyse compliance behaviour Filing 
(causes, options for treatment) * Reporting 
¢ Payment 


Determine treatment strategies 
Plan and implement strategies 


Source: Organisation for Economic Co-operation and Development (OECD), Guidance Note: Evaluating the 
effectiveness of compliance risk treatment strategies 


4.22 The individual (T1) risk-scoring process is designed to estimate 
the amount of tax that may be owed, on a case-by-case basis, using 
information available to the Tl matching process. The accuracy of 
that estimate can be validated only when the taxpayer files a tax 
return, and the actual amount of tax assessed can be compared to the 
estimated amount. One of the Agency’s challenges is that it does not 
know how much tax, if any, is owed until it receives a return. 


4.23 We found that the Agency’s analysis of its T1 file-selection 
process indicated it has some ability to predict the potential of the files 
from non-filers who were assessed after having been identified in the 
matching process. Risk scoring allows the Agency to prioritize returns 
with high potential. For the majority of cases, particularly when the 
file-selection process did not select a case for enforcement action, the 
Agency does not know how well its risk-scoring process works. 
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4.24 The Agency’s research does not provide a reliable estimate of the 
tax potential of the files that the Agency decided not to pursue. To get 
a reliable estimate of the potential of those files, the Agency could 
randomly select a small number of accounts from the 2.5 to 3 million 
individuals that are identified annually as potential non-filers. It could 
then develop baseline measures to determine the effectiveness of its 
risk-scoring process. The Agency has developed a new risk-scoring 
model for personal tax returns and has run pilot projects to test its 
effectiveness. This new model also does not address the potential in 
the files the Agency decided not to pursue. (Our recommendation is at 
paragraph 4.29.) 


The Agency needs to validate its process for registering potential non-registrants 


4.25 The Agency’s process for identifying potential GST/HST non- 
registrants involves matches, leads, projects, and previously identified 
nonr-filers. The Agency focuses on businesses that it believes ought to 
be registered but are not. 


4.26 Its work involving non-registrants uses approximately 

five percent of the NF/NR program resources. Agency guidance 
indicates that when an individual is being reviewed as a non-filer, staff 
must consider and act on GST/HST registration issues at the same 
time. We were unable to determine if this was being done consistently. 


4.27 We looked at businesses that were identified as potential non- 
registrants between | April 2009 and 31 March 2011, to determine 
whether the Agency validated its methods for pursuing non- 
registrants. We found two areas of concern: 


¢ The Agency cannot manage the volume of potential non- 
registrants that its matching process identifies. Each year, this 
automated system identifies around 185,000 potential non- 
registrants and sends their files for review. The Agency has the 
capacity to review about half of these files. This means that a large 
number of potential non-registrants may not be reviewed on a 
timely basis. 


¢ We also noted a file-selection issue. Given the volume of potential 
non-tegistrants identified, it is important for the Agency to select 
files according to their tax potential. While the current method 
includes elements of risk assessment, the Agency has not 
completed analysis that would allow it to determine the 
effectiveness of its file-selection method. 
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Subsection 152(7) of the Income Tax Act— 
A provision of the Act that gives the Agency the 
authority to assess the tax payable where no 
return has been filed by a taxpayer. Agency staff 
calculate an amount they believe is an accurate 
estimate of a taxpayer's income. The taxpayer is 
able to revise the amount of tax assessed by 
filing a corrected income tax return. 


4.28 In summary, the Agency does not know if its selection methods 
to pursue potential GST/HST non-registrants are effective. Given the 
Agency’s limited resources, it is important to prioritize files to pursue. 


4.29 Recommendation. The Agency should determine the 
effectiveness of both its file selection and rejection processes for the 
total population of potential non-filers and non-registrants. 


The Agency’s response. Agreed. The Agency will develop a 
methodology for determining the effectiveness of the file selection and 
rejection processes. It is expected that this methodology will be applied 
to the non-filer/non-registrant population in 2013-14; and, in 2014-15, 
the file selection and rejection processes will be adjusted, as required, 
based on the evaluation of the testing results from previous years. 


File selection for manual pursuit needs improvement 


4.30 Once taxpayer files have gone through the automated process, 
and taxpayers have still not filed a return, the files are transferred to tax 
centres and tax services offices (collectively called “the field”) for 
resolution. Agency field staff will attempt to find current addresses for 
taxpayers, using the Internet and government databases. They may use a 
variety of methods—including phone calls, personalized letters, personal 
visits, and taxpayer lifestyle analyses—to achieve filing compliance. 


4.31 We noted that the Agency has been conducting research to find 
the optimal enforcement strategy for individual non-filers. Using data 
about filing history, the Agency has been trying to identify the non- 
filers that are highest potential and, as such, should be sent directly to 
the field, skipping the step of sending an automated letter. The Agency 
has piloted a new model based on this research, and the initial results 
are encouraging. 


4.32 When field staff pursue a file manually, the potential results are 
* the taxpayer files a return; 


* the Agency assesses the tax payable for the outstanding tax year, 
under subsection 152(7) of the Income Tax Act; or 


¢ the Agency discontinues pursuit of a return, because it determines 
that the file lacks revenue potential or that no return is required. 


In some cases, the Agency may apply penalties or prosecute taxpayers. 
When it has successfully prosecuted a taxpayer, it issues a press release 
to inform the public of the consequences of not filing a return or not 
reporting income. 
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4.33 Guidance from the OECD states that tax authorities should 
ensure that they choose the correct files to pursue, and that they 
consider the cost of getting a non-filer to comply. We looked at 


* how the Agency selects files to pursue manually, 


* whether it evaluated the effectiveness of manual interventions, 
and 


* how it addressed any shortcomings. 


4.34 Tax centres and tax services offices do not have the capacity to 
pursue every potential file transferred from the automated process. 
The Agency has identified an excessive inventory of non-filer cases at 
tax services offices as a significant challenge, so choosing which files to 
pursue is important. 


4.35 Field staff can use the amount of tax potential that the Agency 
estimates to select the potential non-filers that it pursues. They also 
have a number of other tools to help choose files. However, an Agency 
analysis, conducted as part of an NF/NR program renewal exercise, 
determined that the tax potential calculated by the Agency does not 
meet the field officers’ needs. In addition, the analysis found the 
procedures used to select files at the tax services offices varied, as did 
the quality of the file selection. About two thirds of the files selected 
for manual pursuit ended up being discontinued, partly because the 
field staff determined low tax potential after they started to work on 
the file. When a file is discontinued, a decision is made to stop work on 
it, even if the taxpayer has not filed a return. 


4.36 A stronger risk assessment system would allow staff to prioritize 
and focus on files with high potential. It would also provide the 
Agency with the assurance that it can eliminate more low-potential 
files from its inventory and allow it to use its resources more efficiently 
and effectively. 


4.37 Recommendation. The Canada Revenue Agency should 
implement a robust risk assessment system to prioritize its selection 


of files. 


The Agency’s response. Agreed. As noted in this report, the Agency’s 
current effort towards improving file selection through data analysis is 
encouraging. The Agency is committed to developing and applying 
new risk assessment methodologies and business intelligence solutions 
to improve the selection of both low- and high-complexity files. The 
Agency will continue this work in 2012-13, so that a more robust risk 
assessment process can be implemented by 2015-16. 


Report of the Auditor General of Canada—Spring 2012 


Chapter 4 11 


NON-FILERS AND NON-REGISTRANTS—CANADA REVENUE AGENCY 


12 Chapter 4 


The Agency does not pursue all non-filing corporations 


4.38 The Income Tax Act states that, unlike individuals, all corporations 
are required to file a tax return, whether or not they owe taxes. The 
Agency matches corporation business numbers to corporate tax returns. 
If there is no corresponding return, the corporation is a non-filer. 


4.39 We looked at whether the Agency ranks corporations for pursuit 
and excludes corporations based on predetermined criteria. We found 
that corporations are assigned a risk rating based on the estimated tax 
owing. However, the rating is for information purposes only; the 
Agency does not use that rating to select files to pursue. 


4.40 The Agency’s administrative position is that it will not pursue 
certain types of corporations—such as incorporated municipalities and 
non-profit organizations—because they are exempt from income tax. 
Without tax returns for certain types of corporations, it is difficult for 
the Agency to determine if they still meet the conditions for tax 
exempt status. 


Identification projects find non-filers for whom the Agency lacks information 


4.41 The Agency has identified the underground economy as one of 
the biggest risks it faces. The combination of the declining economy 
and a growing number of self-employed individuals increases the risk 
of revenues going unreported and under-reported and of a reduced 
tax base. 


442 According to the Agency, identification projects are one of 
the ways that it identifies and addresses filing and registration non- 
compliance that is missed, or is not prioritized, by the automated process. 


4.43 These projects may be based on various factors, such as local 
knowledge and experience, regional concerns, and national issues. 
The focus of identification projects tends to be on self-employed 
individuals who carry on a business, because the Agency’s automated 
systems address potential non-filers who receive information slips. For 
example, the Agency’s automated matching process would not identify 
a contractor who offers a low “cash-only” price for renovations. 


4.44 Often, the research to identify which businesses to look at begins 
at a tax services office, where staff review a large number of businesses 
for filing compliance. As sources of information, they use the Internet 
and directory and database searches, as well as projects conducted at 
other field offices. They also seek information from other levels 

of government. 


Report of the Auditor General of Canada—Spring 2012 


Report of the Auditor General of Canada—Spring 2012 


NON-FILERS AND NON-REGISTRANTS—CANADA REVENUE AGENCY 


4.45 The Agency’s 2009 Risk Action Plan indicated the need for a 
more horizontal approach to enable a coordinated corporate strategy. 
Furthermore, it indicated that more research, testing, and analysis 
would be critical to help the Agency focus on high-risk industries and 
cases. The Agency’s analysis of completed projects was to be finished 
by early 2011, but it remains outstanding. In addition, the Agency was 
to gather intelligence data, as part of its identification projects. Some 
work has been started, but more is still required to complete the 
intelligence gathering. 


4.46 We looked at identification projects completed between 
April 2009 and March 2011 to determine whether the Agency had 
adequate methods in place to find and compel nor-filers to file 
their returns. 


Examples of projects 


In 2010, the Agency started a project that required a number of cities to provide a 
list of individuals and companies that had received revenues other than salaries. 
This would allow the Agency to check that the recipients’ obligations under the 
Income Tax Act were being met. The information received from the cities would be 
matched against Agency databases. This would allow non-filers and non-registrants 
to be identified and pursued by field officers to file their returns and register their 
businesses. 


Agency staff collected advertisements for home renovations and construction from 
publicly available sources, such as telephone directories, newspaper listings, flyers, 
and websites. This information was used to see whether those businesses filed returns 
and were registered for GST/HST. 


4.47 We found that the Agency undertook some projects during the 
audit period, but its choice of projects was not based on a 
comprehensive analysis of the types of non-filers it might be missing. 
The Agency’s own analysis concludes project activity is ad hoc and has 
not benefited from a more systematic approach. 


4.48 We examined whether the identification projects achieved the 
desired results. We also considered whether the Agency cancelled or 
revised projects that were not meeting intended targets. We found that 
the majority of completed projects met or exceeded their desired 
results. However, we noted that one region was not consistently 
meeting its targets. The Agency has made efforts to address these 
issues at the regional level. 


449 Agency guidance on projects suggests that, when compared to 
other types of field work, identification projects find individuals with 
higher average amounts of taxes assessed. We looked at identification 
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projects and other field work, by comparing the ratio of salaries to 
assessed tax dollars, and we found no difference. This suggests that 
identification projects are no more successful in raising tax revenue 
than any other type of field work. However, they are important to the 
success of the program, because they find the unknown non-filers and 
send an important compliance message to taxpayers. 


Tax services offices rarely follow the new project approval process 


4.50 ‘Tax services offices are expected to spend a portion of their local 
budget on project work each year. They may also request additional 
funding for special compliance projects. In both of the years under 
audit, the Agency budgeted around 15 percent of its field salary dollars 
for identification projects. 


451 The Agency developed a project approval process, with a set of 
rules, for submitting potential projects for approval. It developed the 
process to ensure that it retained knowledge and experience obtained 
during identification projects, and to encourage more rigour at the 
local level when projects are being selected. For example, the process 
identifies the need for 


¢ establishing a business case, 
* performing sample testing, 

* setting objectives, 

¢ establishing a strategy, 

* mapping the process, and 

* having the project approved. 


4.52 We reviewed the applications for project approval by 
Headquarters, to determine whether Agency staff followed the new 
rules, which came into effect in July 2010. We found that most projects 
that were submitted lacked important information. The most common 
omissions were business case and sample testing, as well as approvals 
from local managers. These missing pieces of information are critical to 
demonstrate the need for the project as well as to assist in measuring 
its success. Without sample testing, it is difficult to determine the 
scope of the non-compliance staff are trying to address. Without 
approvals at the local level, managers may be unaware of any issues or 
may disagree with the scope and intent of the project. 
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4.53 With a diverse population to identify and detect, planning 
becomes a key factor in the process of identifying non-filers. Assessing 
what worked in the past, incorporating lessons learned, and 
considering the results of research all play a part in developing a 
business plan. In addition, certain business sectors or geographic areas 
may be more problematic than others. The Agency needs to consider 
what to measure when developing a plan. To ensure success, the 
Agency must set appropriate standards in advance. 


4.54 The overall objective of the Non-Filer/Non-Registrant (NF/NR) 
program is to achieve high levels of filing compliance and registration. 
Managers need to know whether the Agency’s programs and choices 
lead to desired changes in taxpayer behaviour, thus achieving 

this objective. 


The annual planning process focuses on production targets 


4.55 Guidance from the Organisation for Economic Co-operation and 
Development (OECD) refers to the importance of analyzing 
compliance behaviour, determining treatment strategies, and planning 
and implementing those strategies. Agency documents have identified 
non-filers and the underground economy as among the highest risks 
the Agency must address. 


4.56 We reviewed planning documents for the period between 

1 April 2009 and 31 March 2011. We did not find an integrated plan 
that outlines the work of the NF/NR program as a whole. We saw 
budget documents, output targets, and emails. We found that, each 
year, the Agency used the same approach, which focused on obtaining 
a set number of tax returns or registrations and a target amount of 
assessed taxes. Because the Agency did not prepare and evaluate an 
integrated plan, it did not review the program’s overall status. 


4.57 During the years under audit, the internal reports on the 
automated program production had an error in how production results 
were calculated. The Agency has informed us that controls are now in 
place to make sure that these reports are reliable. 


458 In our 2005 audit (November Report of the Auditor General of 
Canada, Chapter 3, Canada Revenue Agency—Verifying Income Tax 
Returns of Individuals and Trusts), we recommended that the Agency 
develop and implement a strategy to better use certain types of 
information returns, such as documents that report the sale of shares. 
This could be another useful source of information in the matching 
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Measuring results and 
making improvements 


process for identifying potential non-filers. The Agency agreed with 
this recommendation but has not yet fully implemented it. 


4.59 The NF/NR program has undertaken a long-term initiative to 
improve its operations. This initiative is based on an internal review of 
the Agency’s offices across Canada, which is the foundation for change 
in areas such as risk management. It is an example of positive steps the 
Agency is taking, through the use of research, to improve its planning 
and program operation. This initiative is in the early stages of 
implementation, and it does not address past recommendations or 
other research agenda issues. 


The Agency considers the cost of achieving filing compliance when 
setting objectives 


4.60 Risk management involves balancing risk with mitigation and 
takes into account the concept of diminishing returns. This requires the 
Agency to focus on obtaining compliance in cases where there is greater 
tax potential, and should consider what resources are necessary to 
achieve that level of compliance. We note, however, that the NF/NR 
program also recognizes the need to send a deterrent message to certain 
sectors, to encourage compliance. 


4.61 Cost-benefit analysis aids in selecting the appropriate approach 
to workload and general planning. We looked at the way the Agency 
considers this analysis when it sets objectives and again, throughout 
the year, when priorities or budgets change. 


4.62 We looked at Agency planning documents and found that, each 
year, it conducted a cost-benefit analysis before it began the automated 
and manual processes to assign non-filing and non-registration files. In 
fact, this concept was the foundation for many of the annual work plans. 


4.63 We reviewed three cases in which additional funds were injected 
into the program partway through the fiscal year. Agency 
documentation demonstrated that a clear cost-benefit analysis was 
conducted in only one of the three cases. 


4.64 The Canada Revenue Agency’s risk management policy, 
guidance from the Organisation for Economic Co-operation and 
Development (OECD), and non-filer/non-registrant (NF/NR) 
operations manuals highlight the importance of regularly learning from 
past experience, relevant research findings, and action plans for the 
purpose of continuous improvement. 
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4.65 In addition to our 2005 audit, we conducted an audit of the NF/NR 
program in 1994 (Report of the Auditor General, Chapter 31, Revenue 
Canada—Ensuring Fairness of the Income Tax System: Detection of 
Non-Filers and Special Investigations). 


4.66 Some of the Agency’s internal audit and evaluation reports 
also addressed the NF/NR program, including the following: 


¢ 2005—Contract Payments Reporting System Program 
Evaluation Study 


¢ 2007—Non-Filer/Non-Registrant Program—Internal Audit Report 
¢ 2008—GST/HST Registration Compliance Evaluation Study 


Performance indicators measure only routine program activities 


467 The OECD notes that revenue collection bodies (organizations 
that administer and enforce national tax laws) have traditionally 
measured their performance through the day-to-day activities that 
result from the various work streams they administer. However, while 
such measures represent an important part of revenue bodies’ reporting 
systems and help with their day-to-day management, they do not offer 
insights into the impact of their activities. Positive trends are 
sometimes assumed to reflect improvements in taxpayers’ compliance, 
but this may not be true. The Compliance Risk Management Process 
(Exhibit 4.2) links taxpayer behaviour to strategies. Strengthened 
performance monitoring would incorporate this concept and help the 
Agency to make this link. 


4.68 Previous audit and evaluation recommendations, which the 
Agency accepted, included the need for performance measures that 
provide more meaningful information. In our 1994 audit report on 
non-filers, we found that the focus on files obtained and taxes assessed 
did not provide information on whether taxpayer behaviour had 
improved. The Agency’s 2008 GST Registration Compliance Study 
found that the yearly production targets that headquarters set for the 
NR workload 


do not demonstrate the extent to which the program is having 
the desired effect of addressing registration non-compliance 
or identifying the relative risk posed by the non-registrant 
population. 


469 The Agency’s 2007 Internal Audit Report concluded that its 
existing program performance measures are sufficient to assess routine 
program activities. However, according to the internal audit, the NF/NR 
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program needed to develop performance measures and indicators to 
accurately report its outcomes. Doing so would provide a complete 
performance picture of its portfolio to support the Agency’s strategic 
decision-making. 


4.70 We reviewed published performance measures, as well as those 

available internally to Agency staff, for the two years under audit. We 
looked for the trend analysis and behavioural impact that were being 
incorporated into management information. 


4.71 We found that the Agency uses the following indicators to assess 
the program 


* amount of taxes assessed, 
* amount spent on salaries, 
¢ number of returns filed, and 


¢ number of GST/HST registrations obtained. 


For the two years under audit, the NF/NR program exceeded its target 
of $2.4 billion for taxes assessed; it assessed $2.8 billion in each of 
the 2009-10 and 2010-11 fiscal years. 


4.12 The Agency also reports the rate of voluntary compliance in its 
Annual Report to Parliament, as a measure of the overall level of filing 
compliance in Canada. Individual filing compliance is quite high; in 
each of the years under audit, 92.8 percent of returns were filed on 
time, exceeding the target of 90 percent. Corporations are less 
compliant; the target of filing 90 percent of returns on time has not 
been met for a number of years (for example, 85.1 percent were filed 
on time for the 2010-11 fiscal year and 85.5 percent for 2009-10). 
(Exhibit 4.3) 


4.73 In its annual report to Parliament for the 2008-09 fiscal year, 
the Agency committed to reporting the percentage of taxpayers 

that it identified as having outstanding tax returns after the Agency 
initiated an action. It did not report these percentages for the 2009-10 
or 2010-11 fiscal years. 


4.74 The Agency does report annually on GST registration 
compliance. However, it reports this compliance as a percentage based 
on GST filing compliance, since the actual number is unknown. In our 
opinion, this is not a meaningful measure because there is no relation 
between those who file their GST returns and those who are not 
registered for GST. The Agency’s annual report for the 2008-09 fiscal 
year indicated that the next report would include a percentage 
measure of the effectiveness of the actions that the Agency took to 
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identify businesses that are required to register for the GST/HST. No 
measure was included in subsequent annual reports. 


4715 The Agency continues to struggle to develop measures that 
demonstrate its effectiveness in addressing filing or registration 
compliance. 


Exhibit 4.3 The Agency’s target and compliance rates 


Percentage f ; 
hes Canadian businesses that were 
registered for the GST/HST 
95 


Individuals, 18 years and older, 


90 BE ok (4): ie CE NB ae ae ee ee who filed their returns on time 
Target level of compliance 


= ae rg ee ee ne Corporations—Taxable incorporated 


businesses that filed their returns 
on time 


2005-06 2006-07 2007-08 2008-09 2009-10 2010-11 


Fiscal years 


Source: Canada Revenue Agency’s annual reports to Parliament, 2005-06 to 2010-11 


4.16 Recommendation. The Canada Revenue Agency should follow 
through on its previous commitments to develop meaningful 
performance measures and indicators. 


The Agency’s response. Agreed. Performance indicators should 
evolve and adapt to changes in the program. Existing indicators 
continue to provide relevant information on program activities, 
expenditures, and results. New indicators, however, can provide even 
more meaningful information if the systems to collect and report on the 
data can be created. System changes will be governed by Agency and 
Government of Canada funding and expenditure policies. Beginning 
in 2012-13, the Agency will, in line with the Auditor General’s 
recommendation, review the relevant performance indicators to 
ensure they further support program reporting requirements. 


The Agency has not integrated research results into planning to improve 
the program 


471 The Agency has been conducting research on nor-filer 
behaviour for many years. Understanding why certain taxpayers do 
not file their returns is essential to developing effective action plans. 


Report of the Auditor General of Canada—Spring 2012 


Chapter 4 | 19 


NON-FILERS AND NON-REGISTRANTS—CANADA REVENUE AGENCY 


20 


Chapter 4 


The Agency has identified some actions it could take to increase 
non-filer compliance, such as changes to the contents of letters sent 
to non-filers (to be implemented for the 2012-13 fiscal year) and 
standard reminders from Agency staff when they respond to taxpayer 
queries about their obligations to file tax returns. 


478 We reviewed the Agency’s use of “data mining” (sifting through 
large amounts of data to find useful information) to improve the way it 
selects cases for non-filer enforcement action. One research project 
looked at the accuracy of the risk assessment when compared to the 
actual amount a non-filer declared on a tax return, regardless of 
Agency intervention. Overall, data mining shows promise. 


4.79 In 1994, we concluded that to conduct compliance research and 
generate more leads, which would help it to find non-filers who are not 
on the tax roll, the Agency needed to continue expanding its use of 
internal and external sources of information. We also concluded that 
the Agency needed to evaluate how effective it was at convincing 
non-filers to file tax returns. 


4.80 We looked at whether Agency enforcement action has an effect 
on voluntary compliance. While the Agency has not conducted any 
research about the effect of its NF/NR program on the Canadian 
public, its research on the non-filer population indicates that 
individuals who were subject to enforcement action were more likely 
to file their returns in future years. However, the impact fades over 
time, and future voluntary compliance is lowest among taxpayers who 
required more severe intervention, such as prosecution. This measure 
could be used to track the impact of the NF/NR program. 


481 We looked at Agency research about non-filers and 
non-registrants to determine whether any of the research findings 
had been integrated into the program planning during the period 
we audited. 


4.82 We found the Agency had a multi-year plan to conduct research, 
but it was not always clear how this plan fit into the NF/NR program’s 
overall plans. We saw no modifications to the annual planning process to 
incorporate research findings; the process was primarily a repeat of the 
previous year’s actions. An Agency’s program evaluation of GST/HST 
Registration Compliance, in 2008, noted similar concerns about risk 
and research. 


4.83 Recommendation. The Canada Revenue Agency should 
include its research findings in its planning and reporting processes, to 
effectively manage the risk of non-filers and non-registrants. 
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The Agency’s response. Agreed. The Agency has made significant 
progress in conducting research and gathering business intelligence 
through the acquisition of data, technology, and the Agency research 
agenda. The Agency recognizes that there are opportunities to 
improve how this intelligence is incorporated into the planning and 
reporting processes. By 2013-14, the Agency will develop and 
implement a formal process to achieve this goal. 


Conclusion 


4.84 The Canada Revenue Agency has adequately addressed non- 
compliance concerning the filing of tax returns and GST/HST 
registration, although we identified some areas for improvement. In 
particular, it does not know the impact of its actions on taxpayer 
compliance. 


4.85 The Agency has identified suitable methods to address filing and 
registration non-compliance. The planning process has some 
weaknesses, because it is not integrated with research and audit 
findings and tends to rely on past methods, although it does include 
the cost of applying particular methods. 


486 The Agency uses a risk-scoring model and data-mining research 
to choose files to pursue through an automated process, but it has no 
way of knowing whether it excludes the right files. Field staff use 
various methods to choose files, in addition to the automated risk- 
scoring system that does not always identify the files with the greatest 
potential for obtaining a tax return and assessing taxes. 


487 Identification projects successfully find non-filers about whom 
the Agency has no previous information. 


4.88 The Agency’s performance measures do not provide information 
about the impact of the Non-Filer/Non-Registrant (NF/NR) program 
or on whether the program met its objective of achieving high levels of 
compliance. Rather, they measure the results of routine program 
activities. In addition, the Agency has made limited progress in 
implementing our previous recommendations or on those from its own 
internal audits and program evaluations. Overall, the NF/NR program 
works, but there is room for improvement. 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these 
standards as the minimum requirement for our audits, we also draw upon the standards and practices of 
other disciplines. 


Objectives 


The overall audit objective was to determine whether the Canada Revenue Agency adequately addressed 
filing non-compliance (non-filers) and registration non-compliance (non-registrants). For the purposes of 
this audit, we defined “adequately” and “addressed.” Adequately meant in a measureable way, with targets 
and indicators reflecting the selection and identification process, and whether the Agency knew at each 
stage (at Headquarters and in tax services offices and tax centres) if the process was effective. Addressed 
meant concentrated on or directed its activities toward. 


We conducted our audit with the following lines of inquiry, each with its own sub-objective: 


- Identifying, selecting, and pursuing non-filers and non-registrants. To determine whether the 
approach to addressing non-filers and non-registrants was implemented as designed, at Headquarters 
and in the tax centres and tax services offices. 


+ Planning. To determine whether the Agency had designed an adequate approach to addressing non- 
filers and non-registrants. 


- Monitoring and reporting actions to address non-filers and non-registrants. To determine whether 
the Agency monitored and reported its actions in addressing the inventory of non-filers and non- 
registrants. 


Scope and approach 


The audit focused on the program areas concerning non-filers and non-registrants with the Taxpayer 
Services and Debt Management Branch. In particular, we looked at how they planned the work, used 
previous audit findings and research, and selected files, according to a risk assessment. We also considered 
how the Agency monitored and reported on the risk of non-filing and non-registration. 


During our audit, we requested and reviewed documents and reports from the Agency. We also analyzed 
data relating to identification projects and file selection. Our review of applications for identification 
projects included the random sampling of 40 files over the two-year period of our audit. We chose our 
sample in accordance with statistical sampling methodology, to provide a high level of assurance that the 
Agency was following procedures when approving projects. A sample size of 34 is sufficient to project the 


entire population of 62 with an expected error rate of 0 percent, a confidence interval of +10 percent, and 
a confidence level of 90 percent. 


To confirm our understanding of the information we received, we interviewed Agency staff at various levels 
and locations. We visited eight tax services offices, three tax centres, and one regional office and conducted 
interviews with regional program advisors, technical advisors, managers, field officers, team leaders, and 
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assistant directors. At the Agency’s headquarters, we met staff in the Enterprise Risk Management Branch, 
Strategy and Integration Branch, and the Taxpayer Services and Debt Management Branch. 


We did not examine Agency activities related to 
* payment non-compliance, 
- non-filing of GST (as opposed to non-registration), and 


* criminal activities of the underground economy. 


Criteria 


The Agency’s plan identifies suitable methods to compel filing 


Corporate Risk Inventory Update, Canada Revenue Agency 
and registration compliance. (CRA), 2010 


Corporate Risk Inventory, CRA, 2009 


Guidance Note: Evaluating the effectiveness of compliance risk 
treatment strategies, Organisation of Economic Co-operation 
and Development (OECD) 


Enterprise Risk Management (ERM) Policy, CRA, 2006 


The plan incorporates past experience, relevant research 
findings, and action plans for the purposes of continuous 
improvement. 


Guidance Note: Evaluating the effectiveness of compliance risk 
treatment strategies, OECD 


Compliance Review II, CRA 


Non-Filer/Non-Registrant (NF/NR) Tax Services Office (TSO) 
Operations Manual 


Enterprise Risk Management (ERM) Policy, CRA, 2006 


The Agency’s plan compares the cost of applying the methods 
designed to mitigate the problem of non-filers/non-registrants 
with the benefit. 


Guidance Note: Evaluating the effectiveness of compliance risk 
treatment strategies, OECD 


Corporate Business Plan 2009-2010 to 2011-2012, CRA 


Guidance Note: Evaluating the effectiveness of compliance risk 
treatment strategies, OECD 


The Agency implements the work plan. 


Canada Revenue Agency Act 


Enterprise Risk Management (ERM) Policy, CRA, 2006 


The Agency identifies and selects inventory in a manner that 
takes into account the cost to achieve filing compliance. 


Guidance Note: Evaluating the effectiveness of compliance risk 
treatment strategies, OECD 


Corporate Business Plan 2009-2010 to 2011-2012, CRA 


Guidance Note: Evaluating the effectiveness of compliance risk 
treatment strategies, OECD 


The Agency validates its method to select inventory. 


The Agency has adequate methods in place to compel non-filers | * Guidance Note: Evaluating the effectiveness of compliance risk 
to file their returns and non-registrants to register their treatment strategies, OECD 
businesses. 


Income Tax Act 


Excise Tax Act 


Canada Revenue Agency Act 
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“The ‘Agency has measures to assess that its actions achieve the ¢ Corporate Risk Inventory Update, CRA, 2010 
eae Westie aaeninsa iby Ue Progr alienlesc ¢ Guidance Note: Evaluating the effectiveness of compliance risk 


treatment strategies, OECD 
¢ NF/NR TSO Operations Manual 


The Agency reports whether its actions achieve the desired Canada Revenue Agency Act 


results as defined by the program objective. Sina encrne 


NF/NR TSO Operations Manual 


¢ Guidance Note: Evaluating the effectiveness of compliance risk 
treatment strategies, OECD 


Management reviewed and accepted the suitability of the criteria used in the audit. 


Period covered by the audit 


The audit covered the period from April 2009 to March 2011. Audit work for this chapter was 
substantially completed on 5 January 2012. 


Audit team 


Assistant Auditor General: Marian McMahon 
Principal: Vicki Plant 
Director: Heather Miller 


Jeff Graham 
Suzanne Moorhead 
Rodney Newcombe 
Stuart Smith 

Sarah Winton 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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Appendix List of recommendations 


The following is a list of recommendations found in Chapter 4. The number in front of the 
recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses 
indicate the paragraphs where the topic is discussed. 


Identifying non-filers and non-registrants 


4.29 The Agency should determine 
the effectiveness of both its file 


selection and rejection processes for the 
total population of potential non-filers 


and non-registrants. (4.20—4.28) 


4.37 The Canada Revenue Agency 
should implement a robust risk 
assessment system to prioritize its 


selection of files. (4.30-—4.36) 


Measuring results and making improvements 


416 The Canada Revenue Agency 
should follow through on its previous 
commitments to develop meaningful 


performance measures and indicators. 


(4.67-4.75) 


Agreed. The Agency will develop a methodology for 
determining the effectiveness of the file selection and rejection 
processes. It is expected that this methodology will be applied to 
the non-filer/non-registrant population in 2013-14; and, 

in 2014-15, the file selection and rejection processes will be 
adjusted, as required, based on the evaluation of the testing 
results from previous years. 


Agreed. As noted in this report, the Agency’s current effort 
towards improving file selection through data analysis is 
encouraging. The Agency is committed to developing and 
applying new risk assessment methodologies and business 
intelligence solutions to improve the selection of both low- 

and high-complexity files. The Agency will continue this work 
in 2012-13, so that a more robust risk assessment process can be 
implemented by 2015-16. 


Agreed. Performance indicators should evolve and adapt to 
changes in the program. Existing indicators continue to provide 
relevant information on program activities, expenditures, and 
results. New indicators, however, can provide even more 
meaningful information if the systems to collect and report on 
the data can be created. System changes will be governed by 
Agency and Government of Canada funding and expenditure 
policies. Beginning in 2012—13, the Agency will, in line with 
the Auditor General’s recommendation, review the relevant 
performance indicators to ensure they further support 
program reporting requirements. 
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4.83 The Canada Revenue Agency 
should include its research findings in 
its planning and reporting processes, to 
effectively manage the risk of non-filers 
and nor-registrants. (4.77-4.82) 


Agreed. The Agency has made significant progress in 
conducting research and gathering business intelligence through 
the acquisition of data, technology, and the Agency research 
agenda. The Agency recognizes that there are opportunities to 
improve how this intelligence is incorporated into the planning 
and reporting processes. By 2013-14, the Agency will develop 
and implement a formal process to achieve this goal. 
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CHAPTER 5 
Oversight of Civil Aviation—Transport Canada 


Performance audit reports 


This report presents the results of a performance audit conducted by the Office of 
the Auditor General of Canada under the authority of the Auditor General Act. 


A performance audit is an independent, objective, and systematic assessment 

of how well government is managing its activities, responsibilities, and resources. 
Audit topics are selected based on their significance. While the Office may 
comment on policy implementation in a performance audit, it does not comment 
on the merits of a policy. ; 


Performance audits are planned, performed, and reported in accordance with 
professional auditing standards and Office policies. They are conducted by 
qualified auditors who 


* establish audit objectives and criteria for the assessment of performance; 
« gather the evidence necessary to assess performance against the criteria; 
* report both positive and negative findings; 

* conclude against the established audit objectives; and 


* make recommendations for improvement when there are significant 
differences between criteria and assessed performance. 


Performance audits contribute to a public service that is ethical and effective 
and a government that is accountable to Parliament and Canadians. 


Table of Contents 


Main Points 


Introduction 


Canada’s civil aviation industry 

Transport Canada’s oversight of civil aviation 
Organizational! structure 

What we found in our 2008 audit 

Current challenge 

Focus of the audit 


Observations and Recommendations 


Regulatory framework 


Transport Canada has implemented a regulatory framework that is consistent with 
international requirements 


Transport Canada’s process for addressing some emerging safety issues is lengthy 
Surveillance planning 

Risk-based planning lacks rigour 

Methodology for planning surveillance lacks clarity on the required minimum surveillance 
Surveillance activities 


Transport Canada has developed a standardized approach for consistent surveillance 
Significantly fewer inspections are done than planned 

Most inspections are not consistently conducted according to established methodology 
Evidence of sufficient management involvement in surveillance activities is lacking 


Human resources planning 


Transport Canada has strengthened its human resources planning process 
Progress has been made to implement human resources strategies, but a lot remains to be done 


Quality assurance 


The Civil Aviation Directorate still does not have an approved quality assurance plan 
Conclusion 
About the Audit 


Appendix 


List of recommendations 
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What we examined 


Why it’s important 


Oversight of Civil Aviation— 
Transport Canada 


Main Points 


Aviation companies that want to operate commercially in Canada 
must obtain an authorization. In 2011, there were more than 

5,000 companies authorized to operate and more than 34,000 aircraft 
registered in Canada. To maintain their authorization to operate, these 
companies must meet the minimum safety standards required for the 
safe conduct of civil aviation. 


The minimum safety standards for aviation companies fall under the 
Aeronautics Act and the Canadian Aviation Regulations. The Act and 
the regulations form the main part of the regulatory framework for civil 
aviation. The overall responsibility for the maintenance of safe, regular, 
and efficient civil aviation operations, including the manufacturing 
and maintenance of aircraft, rests with the aviation industry. 


Transport Canada is responsible for developing and administering the 
policies, regulations, and standards required for the safe conduct of 
civil aviation within Canada’s borders. The Department is also 
responsible for overseeing whether aviation companies have complied 
with this safety framework, and for taking appropriate enforcement 
action where necessary. In 2009-10, Transport Canada spent over 
$148 million and dedicated about 1,400 employees to monitoring civil 
aviation across Canada. 


Our audit examined whether the Department has managed the risks 
associated with overseeing its civil aviation safety program. We focused 
on Transport Canada’s surveillance of air carriers, aircraft maintenance 
organizations, and airports in the National Airports System. 


Audit work for this chapter was substantially completed on 
30 September 2011. Further details on the conduct of the audit are in 
About the Audit at the end of this chapter. 


Transport Canada plays a key role in helping to ensure that Canada’s 
civil aviation safety framework meets minimum international safety 
standards. While Canada compares favourably with many other 
countries in its aviation safety record, any deterioration would 
significantly impact public confidence. This makes it critical that 
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What we found 
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Transport Canada maintain a robust and effective regulatory 
framework for civil aviation safety, especially since the International 
Civil Aviation Organization (ICAO) has projected a significant growth 
in aviation until 2025. 


Identifying aviation companies that present safety risks is a highly 
complex process that relies heavily on the judgment and experience of 
Transport Canada’s inspectors across Canada and on the information 
that is made available to them. 


The Department’s inspectors cannot be continuously present in all 
aviation companies to assess their compliance with aviation safety 
standards. Consequently, Transport Canada must use risk management 
techniques to decide where, when, how often, and in how much depth 
it inspects aviation companies in order to obtain sufficient assurance 
that they are complying with Canada’s aviation safety requirements. 
If Transport Canada does not manage these surveillance risks well, 

it is unlikely to focus its scarce resources On aviation companies and 
operations that represent the highest risks. 


- Transport Canada has developed a rigorous aviation safety regulatory 
framework that is consistent with standards established by the 
International Civil Aviation Organization, but it can take a long 
time to address emerging safety issues—in some cases, more than 
10 years. The Department has yet to fully implement a process that 
will address these issues more quickly. 


Since 2008, Transport Canada has made progress in evolving from 
the traditional surveillance approach—largely based on responding 
to regulatory requirements—to a systems-based approach designed 
for large and small aviation companies. This approach allows for 
more consistent and rigorous surveillance of aviation companies’ 
compliance with safety regulations. 


While some aspects of the new surveillance program are working 
well, there are weaknesses in critical areas. For example, information 
for assessing the risk indicators that Transport Canada uses to 
identify the high-risk aviation companies that should be inspected is 
not always available or kept up to date. A minimum acceptable level 
of surveillance has not been clearly established to indicate how long 
aviation companies can operate without being inspected, and only 
two thirds of planned inspections have been carried out. Most 
inspections are not fully conducted according to established 
methodology and are subject to little management oversight. 
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In addition, documentation of key decisions is weak. Finally, 
Transport Canada lacks a quality assurance program to continuously 
improve its surveillance program. 


Transport Canada recently developed a national human resources 
plan for the oversight of civil aviation to help ensure that it has the 
resources it needs to carry out its safety regulatory program. 
However, the plan does not specify the number of inspectors and 
engineers that are needed, although the Department agreed to 
provide these figures in its response to our 2008 recommendation. 
The Department has made progress with implementing key human 
resources strategies, but efforts to fully implement the new 
surveillance approach have been hampered by the lengthy 
reorganization and by resistance from some inspectors. 


The Department has responded. The Department agrees with all of 
our recommendations. Its detailed responses follow each 
recommendation throughout the chapter. 
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Facts about aviation in Canada 


¢ Canada’s civil aviation industry employs more 
than 90,000 people. 


e Canada has the second largest civil aviation 
aircraft fleet in the world and the second 
largest population of licensed pilots. 
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Introduction 


Canada’s civil aviation industry 


§.1 In 2010, more than 75 million passengers flew within Canada’s 
borders on nearly 3 million air flights. Statistics from the International 
Air Transport Association show that North America has one of the 
safest aviation industries in the world. 


5.2. In 2009 and 2010, the total number of accidents was the lowest 
recorded in a 10-year span in Canada (Exhibit 5.1). Large air carriers 
were involved in very few accidents between 2001 and 2010, and none 
of these accidents resulted in fatalities. These air carriers represent 
more than 95 percent of revenue-generating passenger miles travelled 
in Canada. 


§.3 Canada’s civil aviation industry consists of six major sectors 


(Exhibit 5.2): ' 
* airports and aerodromes, 
* aeronautical product design and manufacturing, 
* aircraft maintenance, 
* air operations, 
* air navigation services, and 


* aviation personnel. 


Exhibit 5.1 Number of accidents between 2001 and 2010 involving Canadian-registered aircraft (travelling inside and outside Canada) and foreign air 
carriers in Canada 


Large Canadian 
air carriers 


Small Canadian 


f f ois) 
air carriers 
Foreign air 


carriers 


Source: Transportation Safety Board of Canada—Statistical Summary, Aviation Occurrences 2010 
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Exhibit 5.2 The sectors that make up Canada's civil aviation industry 


Airports and aerodromes 26 airports in the National Airports 
System, 544 certified aerodromes, and 
1,775 uncertified aerodromes including 
water airports and heliports 


Aeronautical product design and 134 manufacturers 


ea te 1,025 new and modified aeronautical 


products each year 


Aircraft maintenance 1,852 approved maintenance 
organizations 


Air operations 2,260 air carriers 
34,833 registered aircraft 
383 flight training units 


Air navigation services One service provider, 43 air traffic control 
towers, and other facilities 


Aviation personnel 71,137 licences (pilots, flight engineers, 
and air traffic controllers) 


24,000 industry delegates (to certify 
personnel or products on behalf of 
Transport Canada) 


5.4 The aviation industry itself has the primary responsibility for 
maintaining safe, regular, and efficient air travel. Aviation companies 
that want to operate commercially in Canada must obtain an 
authorization to operate. In 2011, more than 5,000 companies were 
authorized to operate in Canada, and more than 34,000 aircraft were 
registered in Canada. 


5.5 According to the International Civil Aviation Organization 
(ICAO), rapid expansion in the aviation industry is making it 
increasingly difficult to sustain a traditional approach to safety 
management, which primarily consists of responding to regulatory 
requirements. In early 2000, ICAO recommended that countries adopt 
a safety management systems (SMS) approach (Exhibit 5.3): 


* Background. The concept of safety management systems 
originated in the early 1980s in the chemical industry. The 
concept emphasized the need to look at an overall process or 
system, including the combination of human, organizational, 
technical, and environmental factors, rather than individual 
occurrences. The goal was for organizations to move from a 
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reactive to a proactive approach by identifying hazards, analyzing 
associated risks, and taking appropriate measures before damage 
could occur. Over the years, the concept spread to other 
industries, including transportation. 


* Evolution. Large civil aviation companies in Canada have put in 
place safety management systems, and large airports are 
transitioning to SMS. Each company must name an accountable 
executive to provide leadership and foster a safety culture, and 
then develop safety policies, procedures, training, and ways of 
managing quality assurance. 


Exhibit 5.3 Transport Canada is adopting a safety management systems (SMS) approach 


Inspector Inspectors are auditors of Inspectors are system 
regulatory compliance. evaluators. As necessary, they 
may conduct traditional audits. 


Company The company responds to The company proactively 
regulatory requirements. manages risks. 


5.6 Canada was the first country in the world to regulate the 
implementation of SMS in the aviation industry. Since 2008, Transport 
Canada has required air operators whose aircraft carry 20 passengers or 
more and their maintenance organizations (referred to as large civil 
aviation companies) to use SMS in managing their safety risks. For 
companies developing and implementing SMS, Transport Canada has 
provided information and help in interpreting the regulatory 
requirements. 


Transport Canada’s oversight of civil aviation 


5.1 Transport Canada is responsible for the oversight of the civil 
aviation industry on behalf of the Government of Canada. Key 
elements of that responsibility include determining an acceptable level 
of aviation safety, and establishing a regulatory framework that sets out 
what the industry must do to achieve and maintain that level of safety. 
Transport Canada delivers certification services that authorize aviation 
companies and personnel to operate based on the requirements set out 
in the regulatory framework. Transport Canada also conducts 
surveillance activities of both aviation companies and their aviation 
personnel to assess whether they comply with the regulatory 
framework. This is done through inspections. 
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5.8 Transport Canada has about 1,400 employees working in civil 
aviation. This represents one quarter of the Department’s workforce. 
Human resources are located in 10 offices across the country and at 
headquarters in Ottawa. In 2010-11, Transport Canada spent 
approximately $148 million on civil aviation. 


5.9 The Department’s mandate for civil aviation derives from the 
Aeronautics Act and the Canadian Aviation Regulations. The Act and 
the regulations form the main pillars of the regulatory framework for 
civil aviation. Under the Aeronautics Act, the Minister has broad 
powers to inspect, audit, and enforce regulations related to any aircraft 
or airport, or any premises used for the design, manufacture, 
distribution, maintenance, or installation of aeronautical products. 


5.10 In 2005, when the Canadian Aviation Regulations introduced 
SMS, Transport Canada began to revise its surveillance activities to 
focus on measuring an organization’s ability to hold itself accountable 
for its own compliance. Transport Canada’s role became one of 
determining if a company’s safety systems complied with the regulatory 
framework (Exhibit 5.4). The Department also looked at how effective 
the company’s safety systems were. As necessary, it could conduct 
traditional audits. 


Exhibit 5.4 Transport Canada's role has changed with the safety management systems (SMS) approach 


Transport Canada inspects aircraft, Transport Canada assesses whether 
records, and personnel directly. aviation companies have effective 
processes in place to ensure safety and 
verifies the company's compliance with 
the regulatory framework. The surveillance 
within the SMS approach may involve 
traditional audits, as necessary. 


Organizational structure 


§.11 ‘Transport Canada carries out its responsibilities for civil aviation 
through the Civil Aviation Directorate at headquarters in Ottawa and 
through Civil Aviation branches in five regions. The Civil Aviation 
Directorate is part of Transport Canada’s Safety and Security Group. 
The Group’s responsibilities include the development of legislation, 
regulations, and national standards to promote safety and security in 
the areas of air, marine, rail, and motor vehicle transportation. 
Additional responsibilities in these areas include monitoring, testing, 
inspecting, and providing limited contribution programs. 
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5.12 Headquarters is responsible for developing the content, policy, 
and standards for oversight activities. Regional Civil Aviation 
branches, along with the National Aircraft Certification Branch, the 
National Operations Branch, and the International Operations Branch 
at headquarters, carry out the oversight activities (Exhibit 5.5). 


Exhibit 5.5 Reporting structure for Transport Canada Civil Aviation Directorate 


Deputy Minister, 
Transport Canada 


Assistant Deputy Minister, 
Safety and Security 


Regional Directors General 
(all transportation modes) 


Director General, 
Civil Aviation Directorate 
(headquarters) 


Regional Directors, 
Civil Aviation branches 
in the regions 


Transport Canada Civil Aviation Directorate 


What we found in our 2008 audit 
5.13 In our 2008 May Report, Chapter 3, Oversight of Air 


Transportation Safety—Transport Canada, we examined how the 
Department had managed the implementation of SMS for large air 
carriers and associated aircraft maintenance organizations, which were 
the first industry groups required to implement SMS. We found that 
Transport Canada’s management of the transition to the new approach 
had several weaknesses, including the following: 


¢ In planning for the transition, the Department did not document 
risks, such as the impact of the transition process on oversight of 
air transportation safety, and did not identify actions to mitigate 
these risks. 


¢ The Department did not measure the impact of shifting resources 
to SMS activities on the frequency of traditional oversight 
activities, which continued for smaller, non-SMS companies. 


¢ The Department had not yet identified how many inspectors 
and engineers it needed, with what competencies, during and 
after the transition. 
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Current challenge 


5.14 The ICAO has forecasted that air traffic volume will likely 
increase in North America by about four percent each year. This 
increase may more than double the current volume of air traffic in 
North America by 2025—the end of the forecast period. If nothing 
else changes, this increase in volume could lead to more accidents. 
The Department recognizes that it will have to do more just to keep 
the accident rate per revenue-generating passenger mile travelled in 
Canada at current levels. At the same time, Transport Canada believes 
that its current funding is not likely to rise significantly during that 
period. Consequently, maintaining the existing civil aviation accident 
rate in Canada will require that the Department effectively manage 
the risks associated with administering its air safety regulatory 
framework and oversight activities. 


Focus of the audit 


5.15 The focus of our audit was to determine whether Transport 
Canada has adequately managed the risks associated with overseeing 
its civil aviation safety program. Risks are inherent in air transportation. 
Eliminating accidents completely is unrealistic, and the Department will 
never have enough resources to ensure that every aviation company in 
the industry complies with all aspects of the safety regulations at all 
times. Therefore, it must use risk management techniques to choose 
what to inspect, when, and how often. It must also rely on properly 
trained staff to use the methodology and tools the Department provides 
them, as well as their knowledge, experience, and training, to identify 
what to inspect and to assess whether aviation companies are complying 
with the regulatory framework. 


5.16 Our audit focused on four aspects of Transport Canada’s 
oversight program: 


¢ First, we looked at whether Canada has a robust regulatory 
framework and whether it is responsive to emerging safety risks. 


* Second, we looked at Transport Canada’s planning of surveillance 
activities and whether they are being conducted according to 
methods in place for that purpose. We examined Transport 
Canada’s surveillance activities of SMS companies, airports in the 
National Airports System (referred to as large airports), and small 
commercial air carriers and maintenance organizations. However, 
we did not examine Transport Canada’s surveillance of other 
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sectors within the civil aviation industry, such as air navigation 
services, flight training units, general/private aviation, or 
personnel licensing. 


¢ Third, we looked at whether Transport Canada has an appropriate 
human resources planning process to help ensure that it has the 
right number of human resources with the right competencies it 
will need to deliver aviation services and surveillance activities. 


* Finally, we looked at whether Transport Canada has a sound plan 
to implement a quality assurance program. 


5.17. Our audit was not designed to conclude on whether air travel is 
safe. That is Transport Canada’s responsibility. We also did not 
examine security issues, the efficiency of surveillance program 
activities, the Department’s activities related to certification and 
licensing, or educational activities that can contribute to improved air 
safety. More details about the audit objectives, scope, approach, and 
criteria are in About the Audit at the end of this chapter. 


Observations and Recommendations 


§.18 Transport Canada is responsible for establishing a regulatory 
framework to oversee civil aviation safety in Canada. The 
International Civil Aviation Organization (ICAO) has developed 
standards and recommended practices (SARPs) to help Canada and 
other member states meet their international obligations related to 
aviation safety oversight. The SARPs set minimum safety requirements 
for a consistent application throughout the world. In addition, Canada 
enters into bilateral agreements with its principal trading partners to 
harmonize safety standards between itself and these countries. 


Transport Canada has implemented a regulatory framework that is consistent with 
international requirements 


5.19 We looked at the regulatory framework for civil aviation safety 
that was in place in 2011 to determine whether Transport Canada had 
implemented a suitable framework to meet its international obligations 
and to address emerging high safety risks. We also examined whether 
Transport Canada had kept abreast of new amendments to [CAO’s 
international SARPs. We focused our work on what Transport Canada 
did to respond to amendments to international SARPs since 2005. 
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5.20 We found that Transport Canada has a comprehensive regulatory 
framework that is consistent with ICAO requirements. In 2005, ICAO 
audited Canada’s civil aviation regulatory framework and similarly 
concluded that it meets Canada’s international obligations. The 
framework consists of three key components: 


¢ The primary law in force is the Aeronautics Act, which establishes 
the Department’s authority to control civil aviation activities 
in Canada. 


¢ The Canadian Aviation Regulations set minimum legal 
requirements, such as those related to the safe operation of air 
carriers, aircraft, airports, and maintenance organizations. 


¢ Transport Canada’s standards set out criteria and conditions to 
provide the industry with a means of compliance and to help the 
industry understand how to fulfill Transport Canada’s expectations 
for meeting a regulation. The Department also develops and uses 
other safety measures, such as educational activities. 


§.21 We also found that Transport Canada has put in place an adequate 
process to keep abreast of changes to international requirements. For 
example, it receives information regularly from ICAO about upcoming 
changes to SARPs and participates in several ICAO committees. [CAO 
senior officials said that Canada is a key participant in discussions about 
any proposed changes to international requirements. 


§.22 ICAO accepts differences related to the international SARPs 
when member states have determined that they are not able to 
implement them, have adopted another solution, or need more time to 
put the new requirements in place. When Canada does not implement 
some SARPS, it must inform ICAO of the differences. Documenting the 
rationale and sharing these differences with ICAO and with the aviation 
industry are important because this allows the industry to make 
informed business and safety decisions. Information on these differences 
is also helpful when Transport Canada negotiates bilateral agreements to 
harmonize safety standards with its main trading partners. 


5.23 We found that Transport Canada systematically compares 
Canada’s regulatory framework with amendments to the SARPs. 

In 2010-11, Transport Canada analyzed 11 ICAO amendments to 
determine the potential impacts on Canada’s regulatory framework 
and filed 63 differences. In all, Transport Canada has filed 

178 differences to ICAO standards in the last five years. However, 
we found that Transport Canada has not effectively kept track of the 
rationale for these differences. In the absence of complete records, 
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it is difficult for Transport Canada to ensure that these differences will 
be considered in the regulatory process when the Department develops 
new regulations. 


Transport Canada’s process for addressing some emerging safety issues is lengthy 


5.24 ICAO requires member states to have a process for the timely 
correction of safety issues. New and emerging risks can come from a 
variety of sources. For example, the Transportation Safety Board 
investigates accidents, reports publicly, issues safety recommendations, 
and raises high safety concerns. Nav Canada reports accidents and 
incidents, which are the basis of entries in the Civil Aviation Daily 
Occurrence Reporting System. As well, Transport Canada has a 
Canadian Aviation Issues Reporting System that allows the public and 
Transport Canada’s employees to report safety concerns. We examined 
whether Transport Canada had a process in place to identify, analyze, 
and prioritize safety issues and emerging high risks, and whether it was 
addressing such risks in a timely manner. 


5.25 We compared emerging safety issues identified by key 
stakeholders with issues compiled by Transport Canada to assess 
whether the Department has an effective process for capturing this 
information. We found that Transport Canada’s process effectively 
identified safety issues through consultations with the civil aviation 
industry and stakeholders, and through participation by Transport 
Canada officials in many national aviation industry groups. 


5.26 ‘Transport Canada has the flexibility to determine whether safety 
issues will be best addressed using regulations or other safety measures, 
such as education, to promote aviation safety. According to the 
Department, the regulatory process is often not the most appropriate 
or cost-effective way to address or solve some emerging safety issues. 
We found that Transport Canada did preliminary analyses of issues and 
determined possible ways to resolve them. In most cases, the 
Department has analyzed practices in other countries. We also found 
that the Department has used tools such as videos, advisory circulars, 
and other means to alert and educate industry and the public about 
these safety issues and how to mitigate them. 


5.21 We found that Transport Canada has recently enhanced the 
criteria it uses to prioritize safety issues. These criteria include factors 
such as the potential impact on international harmonization and key 
findings from the Transportation Safety Board. Transport Canada then 
developed a work plan to address high-priority issues. However, we 
found that the Department has not shared its priorities with the 
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Surveillance planning 


aviation industry and other key stakeholders, and that the time 
between identification of some safety issues and the date when they 
were finally addressed was long—in some cases, more than 10 years. 


5.28 We selected and examined four safety issues raised in past 

years by stakeholders: the state of Canadian runways (1999); pilot 
fatigue (2001); the testing of aircraft wiring and flammability (1999); 
and aircraft collision with land or water (1995). We recognize that many 
steps are required in addressing safety issues, such as consultations, 
analysis of complex regulatory options and their impact on industry and 
international harmonization, and the legislative process itself. However, 
addressing safety issues on a timely basis is critical. Transport Canada 
has not analyzed its process to determine how issues might be addressed 
more quickly. At times, key stakeholders have expressed concern that 
issues they brought to the Department’s attention are not addressed 
quickly enough. Transport Canada’s lengthy process to determine what 
measure is required to address safety issues may cause uncertainties in 
how the industry can respond to those issues. 


5.29 Recommendation. Transport Canada should analyze its process 
for addressing safety issues raised by stakeholders to ensure that it acts 
in a timely way to address significant safety issues. 


The Department’s response. Agreed. By March 2013, the 
Department will have identified ways to accelerate follow-up on 
significant safety issues raised by stakeholders. The new process will 
continue to allow for consultation and regulatory changes. 


5.30 The overall responsibility for aviation safety rests with 

the aviation industry. Even so, the International Civil Aviation 
Organization (ICAO) requires civil aviation authorities, such as 
Transport Canada, to adopt a systematic approach to verifying that 
aviation companies have effective systems for managing the safety of 
their operations. ICAO states that aviation companies should be 
subject to surveillance once every 12 months. 


5.31 Given the size of the aviation industry in Canada, yearly 
inspections would be impractical. Consequently, Transport Canada 
must rely on an effective, risk-based surveillance regime to get the 
assurance that the industry is complying with Canada’s civil aviation 
regulatory framework. We focused our examination of Transport 
Canada’s surveillance activities on its risk-based surveillance plans 

for 2010-11 and on the methodology it used for surveillance planning. 
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Risk-based planning lacks rigour 


5.32 We looked at the information Transport Canada used in 
developing its 2010-11 surveillance plans. Our goal was to determine 
whether the Department had assessed the risk that aviation companies 
do not comply with the civil aviation safety regulatory framework, with 
a focus on high risks. 


5.33 Providing risk assessment guidance. We found that Transport 
Canada’s surveillance methodology requires inspectors and 
management to consider risks in determining how often surveillance 
will occur. In 2009, Transport Canada developed standard risk 
indicators based on an analysis of risk indicators used by other 
international transportation authorities. These risk indicators help 
staff to identify aviation companies that may represent a higher risk for 
non-compliance with the regulatory safety framework. The standard 
risk indicators are comprehensive, covering important risks such as 
changes in key personnel, changes in operations, financial and labour 
difficulties, management practices, safety record, and other risk factors. 
However, Transport Canada’s methodology does not specify what 
information is to be used by inspectors in assessing a company against 
the standard risk indicators. For example, it is not clear what type of 
financial information should be used to assess the risk of a company 
being in “financial difficulty.” 


5.34 Assessing risk. In our discussions with management and 
inspectors, we found that they considered risk indicators when 
assessing each company, but the way they did so varied significantly 
among industry sectors, aviation companies, and regions. 


¢ For example, 75 large air carriers and maintenance organizations 
operate in Canada, including five very large companies that 
account for about 60 percent of revenue-generating passenger 
miles travelled. Transport Canada has assessed the risk indicators 
for the five very large air carriers and associated maintenance 
organizations. For the other 70 large air carriers and maintenance 
organizations, and about 4,000 small air carriers and maintenance 
organizations, the Department did not often use the standard 
risk indicators. 


* Except for the assessment of risks for large airports in one region, 
there were no formal assessments for airports using the standard 
risk indicators when Transport Canada planned its 2010-11 
surveillance activities. 
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Risk profile—A risk profile identifies 


key risk areas, risk assessments and 
tolerance, and an organization's ability and 
capacity to mitigate risks; and 


threats (through ongoing internal and external 
environmental scans, and analysis of risk 
information). 


Source: Treasury Board of Canada Secretariat—Framework 
for the Management of Risk 
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Using the same indicators is essential to ensure that risk assessments 
across the country are consistent and that Transport Canada is 
targeting the right aviation companies for inspections. 


5.35 Compiling risk information. Using up-to-date risk information 
in developing risk profiles is essential to targeting higher-risk 
companies. We found that Transport Canada collects up-to-date risk 
information and prepares risk profiles on the five very large air carriers 
and associated maintenance organizations in planning for inspections. 


5.36 However, for most large air carriers and maintenance 
organizations, for large airports, and for small air carriers and 
maintenance organizations, Transport Canada is missing key risk 
information and has no formal process in place to collect that data. 
Transport Canada collects a great deal of information, including 
information from previous inspections, companies’ operating manuals, 
and incidents reported in the Civil Aviation Daily Occurrence 
Reporting System, as well as other information on companies that 
transport or handle dangerous goods. However, that information is 
stored in different databases and is difficult to access when regional 
civil aviation branches are assessing the risk indicators of these 
companies. In addition, Transport Canada has little or no information 
on some important risk indicators. For example, information on the 
financial situation of aviation companies is often not collected. As a 
result, decisions on risk-based planning are often based solely on the 
experience of the few inspectors involved with those entities in the 
past. With recent organizational changes to adapt to the safety 
management systems (SMS) approach and turnovers of inspectors, it is 
even more important to ensure that the risk information ‘used in 
making planning decisions is complete and reliable. The problem is 
particularly acute with aviation companies and large airports that were 
not inspected in the previous year. Without complete and reliable risk 
profiles to conduct risk assessments, the Department may not inspect 
the aviation companies that present the highest risks to aviation safety. 


5.37. Recommendation. Transport Canada should clarify what 
information on industry and aviation companies should be used in 
making risk-based decisions, collect that information, assess its 
completeness and reliability, and develop risk profiles when preparing 
annual surveillance plans in the regions. 


The Department’s response. Agreed. Transport Canada is committed 
to maintaining Canada’s very strong civil aviation safety record. It has 
developed and begun implementation of a comprehensive action plan 
to guide the modernization of its structure and improvements to its 
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¢ SMS companies: an inspection every year— 
an assessment of the entire SMS every 
three years and a targeted inspection each of 
the other two years. 


¢ Non-SMS companies: an inspection of key 
functional areas once a year. 
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systems, processes, and tools, and to ensure that resources are targeted 
to areas of highest risk. The Assistant Deputy Minister of Safety and 
Security is accountable for the implementation of this plan, and its 
progress is being monitored by a senior management steering 
committee chaired by the Associate Deputy Minister. Successful and 
complete implementation of the plan is a priority for the Department. 


By August 2012, the Department will complete and launch a standard 
risk assessment tool. The new tool is designed to collect information 
necessary to assess aviation companies and ensure consistency across 
all aviation enterprises, with regularly updated risk profiles to support a 
dynamic process of surveillance. These steps will improve the 
information base for the development of annual surveillance plans and 
overall surveillance activities. 


Methodology for planning surveillance lacks clarity on the required minimum 
surveillance 


§.38 For planning surveillance activities, Transport Canada must 
determine a minimum level of surveillance and manage its surveillance 
risks to an acceptable level. Setting clear minimum surveillance 
requirements for aviation companies is important if Transport Canada 
is to be sure that aviation companies are conducting safe operations, 
and if the Department wants to demonstrate that it is managing its 
surveillance risks. The safe conduct of operations is largely assured 
when companies comply with the terms of their authorization to 
operate. We looked at Transport Canada’s 2010-11 surveillance plans 
and interviewed those involved in their preparation to determine 
whether the Department had respected its own planning guidance on 
how often individual aviation companies are inspected. 


5.39 We found that Transport Canada has developed national 
planning frequencies that require aviation companies to be inspected 
annually. This requirement is consistent with ICAO standards. 
According to the Department’s own planning methodology, inspectors 
are also required to consider risk information and availability of 
resources when developing their annual surveillance plans. 


5.40 We also found that although the national planning frequencies 
state that aviation companies must be inspected every year, about 

70 percent of aviation companies across the country were not 
inspected in 2010-11. Transport Canada’s planning frequencies also 
state that the decision to extend the frequency of surveillance beyond 
one year should be supported by a risk assessment. We found that 
management did not analyze and document its rationale for 
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Surveillance activities 


postponing the planned annual surveillance. This practice is important 
to ensure that high-risk companies are inspected each year and that 
the correct level of management is aware of the decision to postpone 
the surveillance of high-risk companies and agrees with it. We also 
found that the planning methodology does not provide guidance on 
how long a company is permitted to operate without being inspected. 


5.41 Recommendation. Transport Canada should clarify the 
minimum level of surveillance needed to provide the necessary 
coverage of civil aviation companies. 


The Department’s response. Agreed. By August 2012, Transport 
Canada will update and communicate to all inspectors and managers 
minimum surveillance levels for civil aviation companies. 


Transport Canada has developed a standardized approach for consistent 
surveillance 


5.42 The International Civil Aviation Organization (ICAO) requires 
member states to provide guidance and tools to staff who perform 
oversight activities. In our 2008 report on Transport Canada’s 
oversight of air transportation safety, we observed that Transport 
Canada had not explained how the day-to-day work of inspectors 
would change as activities related to safety management systems 
(SMS) were integrated with traditional oversight activities. 

Since 2008, Transport Canada has developed a new methodology for 
carrying out its surveillance activities. The methodology applies to all 
aviation companies authorized to operate and was designed to be 
flexible enough to apply to SMS companies as well as the other 
aviation companies. We looked at Transport Canada’s surveillance 
methodology to determine whether it was adequate. 


5.43 We found that Transport Canada has developed a standardized 
surveillance methodology designed to promote consistent and rigorous 
inspections across Canada, and to make it possible for Transport 
Canada to determine a company’s compliance with regulations. 

The methodology provides detailed staff instructions for routine 
surveillance (surveillance staff instructions SUR-001), as well as 
guidance and audit tools to carry out the surveillance. 


5.44 We examined the instructions to staff for conducting routine 
surveillance activities. We found that the instructions clearly define 
the Department’s expectations, set out the roles and responsibilities of 
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key players involved in routine surveillance activities, and include key 
elements of good surveillance methodology for 


* preparing and completing inspection plans, 
* selecting sampling parameters, 
* identifying procedures on how to conduct the inspection, and 


¢ drafting reports and communicating inspection results to the 
aviation companies inspected. 


5.45 However, we found that the instructions lack the necessary tools 
to document the results of inspection work. As well, the instructions 
provide little guidance on the minimum documentation needed to 
support the key judgments made by inspectors in preparing, 
conducting, and reporting inspection findings. Finally, the instructions 
lack guidance about how management should be involved in reviewing 
those decisions and how their involvement should be documented. 
Having sufficient and appropriate evidence to support inspection 
findings and having documented management reviews would help to 
demonstrate that inspectors are following the methodology and that 
the desired level of assurance is being achieved. 


5.46 Recommendation. Transport Canada should review its staff 
instructions to clarify its expectations on the minimum documentation 
required and the level of management involvement expected in the 
oversight of surveillance activities. 


The Department’s response. Agreed. By December 2012, the 
Department will have completed an update of, and communicated to 
managers and staff clear direction on, minimum requirements for 
surveillance documentation and management involvement. 


Significantly fewer inspections are done than planned 


5.47 Transport Canada’s national planning frequencies require that 
the higher-risk companies be included in annual surveillance plans. 
We looked at the actual surveillance activities conducted in 2010-11 
to determine whether Transport Canada had completed its inspections 
as planned. We found that only 67 percent of air carriers, maintenance 
organizations, and large airports were inspected according to annual 
surveillance plans. 
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5.48 This result is significant because only the higher-risk companies 
are to be selected for inspection and included in the annual surveillance 
plans. If these plans constitute the minimum level of inspection that 
management believes is necessary to provide assurance that companies 
are complying with the safety regulations, then Transport Canada 
cannot have the assurance it says it needs. Even if more than the 
minimum number of inspections is planned, management still needs to 
assess whether, on the basis of the inspections done, Transport Canada 
has obtained an appropriate level of assurance. 


5.49 We were told that the main reasons for cancelling or postponing 
inspections were to reallocate resources to other higher-risk 
inspections, or because planned resources were not available. However, 
we observed that the rationale for moving resources from planned 
inspections to other work or to other inspections was not documented 
and was not approved by senior management. We also questioned why 
planned resources were not available in 2010-11. We found that 
Transport Canada had overestimated the time inspectors had available 
to conduct surveillance activities. We also found that the Department 
did not have accurate records on the time inspectors spent on various 
oversight activities. In the absence of such information, it is difficult 
for the Department to develop realistic surveillance plans for achieving 
a minimum level of surveillance activities. 


5.50 Recommendation. Transport Canada should determine why a 
number of inspections are falling short of plans and take corrective 
action. Deviations from original plans should be documented, and they 
should be approved by senior management. Senior management 
should also determine whether Transport Canada is still obtaining the 
desired level of assurance that companies are complying with aviation 
safety regulations. 


The Department’s response. Agreed. Minimum surveillance levels 
will be reflected in surveillance plans from 2012-13 forward. 
Performance will be monitored on a regular basis, any deviations from 
approved levels will require respective Regional Director General and 
Director General Civil Aviation approval nationally, and senior 
management will be provided with regular updates. 
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Most inspections are not consistently conducted according to established 
methodology 


9.91 ‘Transport Canada inspectors are responsible for inspecting 
aviation companies against Canada’s regulatory framework. This 
activity requires a good understanding of the methodology they must 
use to prepare for, conduct, and report on surveillance activities. The 
surveillance process relies on the judgment of Transport Canada’s 
inspectors and on their training and experience to analyze and validate 
the safety systems and practices of aviation companies. 


5.52 We randomly selected and reviewed 74 inspection files 
completed in 2010-11. They included inspections of large and small 
commercial air carriers and maintenance organizations, as well as large 
airports. We reviewed these files to determine whether inspection 
teams had conducted their surveillance according to the established 
methodology. We did not examine the inspectors’ judgments. Rather, 
we reviewed the documentation provided to us and met with Transport 
Canada inspectors. 


5.53 In our review of files, we found examples of good surveillance 
practices and documentation for inspections done at the larger 
aviation companies that concluded on the company’s compliance to 
the regulatory framework. However, we found that most inspections 
were not consistently conducted according to the surveillance 
methodology. In discussion with inspectors, we often found that they 
did not fully understand how to apply the new methodology. 


5.54 Preparation for the inspection. The instructions require 
inspectors to review key documentation and prepare an inspection 
plan, including a sampling plan, before they inspect aviation 
companies. This is important to ensure, among other things, that 
inspections focus on highest-risk areas and provide enough assurance 
that the company is operating within the regulatory framework. 


5.55 We found that inspection plans were prepared before the 
inspection work began, in about 35 percent of the files that we 
reviewed. These plans included some of the key elements, such as the 
scope of the inspection, the team members, the inspection schedule, 
and a list of interviews. However, we found little information on the 
key tests that had to be done to ensure that the inspection would focus 
on the greatest risks. We also found some information on sampling, but 
sampling plans were rarely prepared. In the absence of a sampling plan, 
Transport Canada may not know whether enough records were tested 
and enough interviews were conducted to conclude on the aviation 
company’s compliance with the regulatory framework. 
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5.56 Documentation practices. Good file documentation is 
important to facilitate inspection reviews by management, to show 
that enough work has been done to support inspection conclusions on 
the aviation company’s compliance with the regulations and to support 
any enforcement action that might be needed. 


5.57 In the files that we reviewed, we found that the inspectors had 
conducted interviews, reviewed records, analyzed documentation, 
identified issues, and concluded on whether the company was complying 
with the regulations. However, because there are no requirements for 
minimum documentation of work done and reporting of inspection 
results, the quality of the documentation varied significantly among 
inspectors and across regions. We found examples of documentation on 
interview strategies, the extent of company records reviewed, minutes of 
interviews, and actual company documentation reviewed and results of 
that review. However, this was not the case in most files. 


5.58 Staff instructions require inspectors to assess whether their 
findings are systemic and whether they could have a significant impact 
on safety. However, we found that these assessments were usually not 
documented. In some cases, the findings in the report were different 
from those on the inspection worksheets. In many other cases, 
findings on the worksheet were not included in the inspection report. 
Accurate and complete reporting of significant problems is key to 
helping aviation companies undertake corrective action to resolve 
them. As well, we found that Transport Canada had trouble locating 
key inspection documents, such as inspection reports. In several cases, 
when these documents were found, they were only in draft form. In our 
view, Transport Canada’s documentation practices are weak and could 
be significantly improved. 


5.59 Recommendation. Transport Canada should ensure that 
inspections are conducted and documented consistently according to 
established methodology. It should also consider whether existing 
documentation tools are sufficient or need to be redesigned or replaced. 


The Department’s response. Agreed. A review of tools and the 
launch of improved tools will be completed by December 2012. 
Guidance and new tools to improve documentation of inspections will 
be provided to staff as they are approved. This documentation will be 
reviewed regularly as part of Quality Assurance activities. 


9.60 Inspector training. Inspectors receive a delegation of authority 
to act on behalf of the Minister in performing certain duties, and they 
must be trained accordingly. In 2008, Transport Canada agreed with 
our recommendation that a training strategy should be developed to 
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align with its human resources plan and address required 
competencies. In this audit, we looked at whether the required training 
had been provided to the delegated inspectors. 


5.61 We found that the mandatory training for all inspectors is 
identified in a departmental directive. It includes courses in support of 
the new surveillance approach, such as an introduction to the 
concepts and principles of SMS, proactive interview skills, an overview 
of quality assurance, and surveillance procedures. We also found that 
the Department has implemented a process to track the training 
provided to all inspectors. While this training covers most of the 
essential aspects of the inspectors’ work, it does not include courses on 
working as a team—a theme that Transport Canada identified as 
important in the human resources plan that supports implementation 
of its new surveillance approach. 


5.62 By the end of March 2011, a majority of inspectors had received 
mandatory training on the concepts and principles of SMS, on 
proactive interview skills, and on quality assurance. However, at that 
time, only 40 percent of inspectors had been trained on the new 
surveillance methodology. As a result, we noted that many inspections 
were carried out in 2010-11 by inspectors who had not received this 
training. Completing the training on time is important to help 
inspectors understand and apply the new surveillance methods. 
Otherwise, the Department will not have the assurance it needs that 
aviation companies are complying with air safety regulations. At the 
end of our audit, more than 65 percent of inspectors had completed 
the training on surveillance procedures. 


5.63 Recommendation. Transport Canada should ensure that all staff 
involved in inspections are trained in a timely manner so they can 
carry out their responsibilities. 


The Department’s response. Agreed. By July 2012, all current 
inspectors will have been given surveillance procedures training. There 
will be regular management reviews to ensure that mandatory training 
requirements continue to be met and that evolving training needs 

are addressed. 


Evidence of sufficient management involvement in surveillance activities is lacking 


5.64 Staff instructions state that inspection plans and reports should 
be reviewed and approved by a supervisor. Management review of 
inspection plans and subsequent adjustments to the plans are 
particularly important given the need to focus scarce resources on 
the highest risks. Review is also important to assure management that 
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Human resources planning 


inspections were completed as planned and that the evidence collected 
supports the conclusions reached. 


5.65 However, we found that Transport Canada does not have a 
formal management review and approval process to assess the quality 
of inspections or compliance with the surveillance methodology. In 
most cases, we found that important planning decisions, such as 
inspection plans and the key judgments upon which they were based, 
had not been approved by management. We also found that while 
some inspection reports for larger aviation companies had been 
reviewed and approved by management, the practice itself and the 
extent of those reviews varied significantly among regions. The fact 
that the documentation of inspections is weak, that many inspectors 
still have not received the required training as of 31 March 2011, and 
that aspects of the approved methodology are not being followed 
points to a lack of management review and approval in overseeing the 
surveillance program. 


§.66 ‘Transport Canada informed us that it plans to fully implement 
an integrated management system in 2013 to guide and support 
management decisions in civil aviation oversight. In developing such a 
system, it needs to provide for appropriate management involvement 
in surveillance activities. 


5.67 Recommendation. Transport Canada should develop a clear 
expectation for appropriate management review and approval for 
planning, conducting, and reporting on surveillance activities to 
ensure that staff comply with established methodology. The nature, 
timing, and extent of that involvement should be documented. 


The Department’s response. Agreed. By June 2012, the Department 
will put in place a formal process to ensure appropriate management 
review and sign-off of surveillance reports. 


5.68 In our 2008 audit, we looked at whether the Department had 
enough inspectors with the right skills and competencies, in the right 
place at the right time, to carry out its mandate. We also looked at 
whether the Department had an integrated human resources plan for 
civil aviation that was aligned with its strategic plan. We focused our 
audit work on how Transport Canada was managing the transition to 
safety management systems (SMS) for the large aviation companies. 
We reported that Transport Canada had not yet identified how many 
inspectors and engineers it needed, with what competencies, during 
and after the transition. We also reported that the impact of SMS was 
being addressed in the reorganization of the Department’s civil 
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aviation safety program, to be completed before the end of 2009. 

We issued three recommendations to strengthen human resources 
planning: to identify future staff needs; to implement recruitment and 
training strategies to meet those needs; and to restrict authority if 
inspectors did not take the required training. The Department agreed 
to implement our recommendations. 


5.69 In this audit, we looked at whether Transport Canada has 
adequately planned the human resources it will need to deliver its civil 
aviation safety oversight, and whether it had addressed our 

2008 recommendations. We interviewed Transport Canada officials 
responsible for human resources planning; met with inspectors; and 
reviewed key planning documents, including the September 2011 
transition plan for improving civil aviation safety, the 2009-12 national 
human resources plan for civil aviation, and the 2010 strategic plan for 
civil aviation. We also conducted interviews with senior management 
about strategies that were being implemented and their level of 
achievement, as outlined in the Human Resources Plan. 


Transport Canada has strengthened its human resources planning process 


5.70 We examined the current human resources planning process to 
assess whether it was aligned with the approach recommended by the 
Treasury Board of Canada Secretariat. We also examined the resulting 
plan to determine whether it addressed the recommendation we made 
in 2008 that the plan should specify the number of inspectors and 
engineers and the competencies they would need. 


5.71 We found that the planning process followed by Transport Canada 
to prepare a human resources plan was consistent with the approach 
recommended by the Secretariat. It included an analysis of the civil 
aviation environment and priorities that are aligned with its strategic 
direction. The planning process involved extensive consultations among 
staff representatives at headquarters and in the regions and with senior 
management. It also took into account regional human resources needs. 


5.72 Asa result of the planning process, the Department identified 
gaps in certain competencies and proposed strategies to fill these gaps. 
For example, Transport Canada has acknowledged that bringing 
together and better integrating inspectors’ skills and knowledge are 
essential to achieving its objectives. Because most large aviation 
companies perform multiple activities like flight operations and 
maintenance, and have integrated systems and practices, inspectors 
are required to work more in multi-disciplinary teams that mirror the 
makeup of the aviation companies they regulate and oversee. 
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5.73 However, while the Department identified some gaps in 
competencies in its human resources plan, it did not identify how 
many inspectors and engineers it will need to conduct inspections 
under the new surveillance approach envisioned by the plan. As we 
noted in 2008, it is important that the Department establish the 
number of resources it will need to achieve its strategic objectives and 
to develop an appropriate strategy to obtain them. 


5.74 Recommendation. Transport Canada should identify the 
number of resources and the competencies it will need to plan and 
conduct inspections under its new surveillance approach, and develop 
a strategy to obtain them. 


The Department’s response. Agreed. The 2012-13 surveillance 
plan will be used to determine the number of resources and the 
competencies required by the Department. This information will 
inform updates to the national human resources plan for civil aviation, 
which will include strategies for recruitment and training. 


Progress has been made to implement human resources strategies, but a lot 
remains to be done 


5.75 The human resources strategies to address the gaps in 
competencies identified in the Human Resources Plan include the 
rewriting of work descriptions, the reclassification of positions 
(including those of inspectors) to support the new organizational 
structure, the development of competency profiles, and the delivery of 
courses to address training needs. We conducted interviews with senior 
management and found that the Department has made important 
progress toward the implementation of its human resources strategies. 
For example, the Department has widely consulted with staff to design 
the new organizational structure and validate work descriptions for 
inspectors. It has also put in place a new organizational structure at 
headquarters and in the regions. However, addressing these issues is 
complex, and a lot remains to be done to fully implement the human 
resources strategies. Transport Canada still has to finalize job 
descriptions for about 55 percent of positions and the classification 
decisions related to them. In addition, at the time of our audit, the 
Department still needed to deliver training on the new surveillance 
methodology to 35 percent of inspectors. Although Transport Canada 
has made it a priority to speed up its progress in achieving these 
strategies, the six-year period it has taken to implement the 
reorganization and the ongoing resistance among some inspectors have 
hampered the Department’s efforts to fully implement the new 
surveillance program. 
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The Civil Aviation Directorate still does not have an approved quality assurance plan 


9.16 ‘Iransport Canada’s 2008 Civil Aviation Directive requires 

that the Department plan and conduct internal quality assurance 
assessments within its directorate. An effective quality assurance 
program for evaluating Transport Canada’s surveillance activities and 
oversight services would set high-level direction on quality expectations, 
would help management determine whether established methodologies 
are being followed, and would facilitate continuous improvement. 


5.77 In our 2008 audit, we reported that Transport Canada had no 
national mechanism for monitoring consistency in oversight activities 
or risk assessments. We recommended that Transport Canada establish 
a national mechanism to provide the desired level of assurance that 
policies, procedures, and processes for civil aviation oversight 
activities, including the assessment of risks, are followed consistently 
in all regions. 


5.78 While we were scoping this audit, Transport Canada told us that 
it had not yet implemented a quality assurance program. Based on this 
information, we focused our work on the lesser expectation that 
Transport Canada had an approved plan to implement an adequate 
quality assurance program to promote continuous improvement of civil 
aviation safety. 


5.79 We found that Transport Canada had briefly introduced some 
quality assurance assessments after our 2008 report, but put them on 
hold later that year to deploy resources elsewhere in the Department. 
We also found that Transport Canada conducted some quality control 
exercises, but these were limited to reviewing the quality of inspection 
reports in some regions. 


5.80 At the time of our audit, Transport Canada was working on terms 
of reference for developing its quality assurance plan. The Department 
stated that it would develop a national risk-based quality assurance plan 
by early 2012 and put quality assurance assessments in place by spring 
2013. A properly functioning quality assurance program would help to 
identify the weaknesses we found in how the Department prepares for, 
conducts, and reports on its surveillance activities. 


5.81 Recommendation. Transport Canada should develop a detailed 
quality assurance plan and make it a priority to implement a rigorous 
quality assurance program that is consistent with best practices. 


The Department’s response. Agreed. The Department is working on 
a rigorous quality assurance plan that reflects updated surveillance and 
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inspection methodology, which will be fully completed and 
implemented by March 2013. Results of quality assurance work will 
be used to improve the Department’s work related to its oversight 
responsibilities. 


Conclusion 


5.82 Transport Canada has shown leadership in moving Canada 
toward a regulatory framework that includes a safety management 
systems (SMS) approach. Its actions were designed to promote civil 
aviation safety in Canada, with the objective of maintaining our good 
safety record into the future. As the first member of the international 
community to move to SMS, it faced significant challenges without the 
benefit of others’ experience. Transport Canada has made progress in 
adapting its regulatory framework to one based on SMS and in 
developing a surveillance methodology that is in line with an SMS 
approach to civil aviation safety. 


5.83 We concluded that Transport Canada has implemented a 
suitable regulatory framework for civil aviation safety. The framework 
is consistent with Canada’s international obligations, and the 
Department has the systems and practices necessary to improve it in 
response to emerging safety risks. 


5.84 Ona program-wide basis, however, we concluded that Transport 
Canada is not adequately managing the risks associated with its civil 
aviation oversight. The significant weaknesses that need to be 
addressed involve how the Department plans, conducts, and reports on 
its surveillance activities. While we found examples where surveillance 
activities met our audit criteria, most inspection files that we reviewed 
fell short and did not follow Transport Canada’s own established 
methodology. We found limited evidence of management review and 
involvement in surveillance activities. We also found that many fewer 
inspections are done than planned. This is significant considering that 
only the companies and the operational areas considered to be of 
higher risk are selected for inspection in any given year. 


9.85 We also concluded that Transport Canada has not adequately 
planned the human resources it will need to deliver its civil aviation 
safety program. It has developed a national human resources plan and 
made progress in implementing key human resource strategies, but it 
has not identified how many inspectors and engineers are needed to 
oversee civil aviation safety. 
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5.86 Finally, we concluded that Transport Canada does not have an 
approved plan to implement an adequate quality assurance program 
that will promote continuous improvement of its civil aviation 
safety program. 


5.87 For decades, Transport Canada’s approach to overseeing civil 
aviation safety remained unchanged. The Department faced 
challenges as the first nation to move to an SMS-based approach in 
the aviation industry. It has made real progress, with large aviation 
companies now working in the SMS regulatory framework. Transport 
Canada has revised its surveillance methodology so it is consistent with 
this new approach, and inspections are being carried out under its 
instructions. Senior management now needs to concentrate its efforts 
on ensuring that staff apply the approved methodology consistently 
and rigorously, that managers provide the necessary review and 
supervision, and that an effective continuous improvement program is 
put in place. Otherwise, Transport Canada will not have the assurance 
it needs that the industry is operating in compliance with the 
regulatory framework for civil aviation safety in Canada. 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these 
standards as the minimum requirement for our audits, we also draw upon the standards and practices of 
other disciplines. 


Objectives 


The overall audit objective was to determine whether Transport Canada has adequately managed the risks 
associated with overseeing its civil aviation safety program. 


The audit sub-objectives were 


* to determine whether Transport Canada has implemented a suitable civil aviation safety regulatory 
framework; 


* to determine whether Transport Canada has appropriately monitored the compliance of air carriers, 
maintenance organizations, and large airports against the civil aviation safety regulatory framework, 
with a focus on high safety risks and by using an adequate surveillance methodology; 


* to determine whether Transport Canada has adequately planned the human resources it will need to 
deliver its civil aviation safety program; and 


* to determine whether Transport Canada has a plan to implement an adequate quality assurance 
program to promote continuous improvement of civil aviation safety. 


Scope and approach 


Our audit focused on branches in three regions and at headquarters that are involved in the delivery of 
Transport Canada’s civil aviation safety program. We examined implementation of the regulatory 
framework and how Transport Canada was managing its risks related to its surveillance responsibilities. 


We selected and reviewed significant safety issues and emerging high safety risks raised by stakeholders to 
assess how Transport Canada has monitored and responded to them. 


We examined the information used for planning surveillance decisions and the process followed by 
Transport Canada for planning its annual surveillance activities in 2010-11. We also examined the 
methodology used by inspectors in 2010-11 for conducting planned surveillance activities. We randomly 
selected 74 inspection files from 2010-11 from SMS companies, airports transitioning to SMS, and small 


air operators and maintenance organizations. More specifically, we randomly selected the following in 
three regions: 


* a representative sample of 52 of 375 inspection files for small air carriers and maintenance 
organizations, 


- 6 large airports of the 10 airports in the National Airport System, 
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- 6 of 14 inspection files for large air carriers, and 


- 6 of 13 inspection files of maintenance organizations. 


At headquarters, we randomly selected 2 of 5 inspection files of very large air carriers and 2 of 5 inspection 
files of maintenance organizations. 


Where representative sampling was used, sample sizes were sufficient to conclude on the sampled 
population, with a margin of error of 10 percent, 18 times out of 20. We did not examine the inspectors’ 
judgments or their competency. Rather, we examined whether inspectors conducted surveillance activities 
according to the established surveillance methodology. 


Finally, we looked at Transport Canada’s plans for planning human resources and quality assurance 
activities. 


We collected evidence through interviews with Transport Canada’s officials at headquarters in Ottawa and 
at several regional offices. We met with representatives from the civil aviation industry to obtain an 
understanding of the industry. 


We did not examine the safety or efficiency of the aviation industry. 


Criteria 


Transport Canada has implemented a suitable civil aviation Safety Oversight Manual, second edition, International Civil 
safety regulatory framework to address significant safety issues Aviation Organization (ICAO), 2006 


and emerging high risks in a timely manner. Safety Management Manual, ICAO, 2009 


“Transport has assessed the risk of non-compliance ¢ Safety Management Manual, ICAO, 2009 
against the civil aviation safety regulatory framework, with a 
focus on high safety risks. 


Framework for the Management of Risk, Treasury Board of 
Canada Secretariat, 2010 


Transport Canada has planned its risk-based annual surveillance | * Safety Oversight Manual, second edition, ICAO, 2006 
according to its surveillance methodology. 


Transport Canada has conducted its surveillance activities ¢ Safety Oversight Manual, second edition, ICAO, 2006 
according to established surveillance plans and methodology. 


Transport Canada has developed a human resources planning Integrated Planning Guide, Treasury Board of Canada 
process so that it has the appropriate number of qualified staff Secretariat, 2007 
needed to deliver aviation services and surveillance activities. 
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“Transport Canada has developed a plan to implement an ¢ International Organization for Standardization ISO 9001, 
adequate quality assurance program to promote continuous 2008 
improvement of its civil aviation safety program. 


Management reviewed and accepted the suitability of the criteria used in the audit. 


Period covered by the audit 


The audit period examined was April 2010 to March 2011. Audit work for this chapter was substantially , 
completed on 30 September 2011. 


Audit team 


Assistant Auditors General: John Rossetti and Maurice Laplante 
Principal: Régent Chouinard 

Lead Director: Lucie Talbot 

Directors: Chantal Thibaudeau and Nadine Cormier 


Marie-Claude Dionne 


Audrey Garneau 
Michelle Gorman 
Sophia Khan 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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The following is a list of recommendations found in Chapter 5. The number in front of the 
recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses 
indicate the paragraphs where the topic is discussed. 


Regulatory framework 


5.29 Transport Canada should analyze 
its process for addressing safety issues 
raised by stakeholders to ensure that it 
acts in a timely way to address 
significant safety issues. (5.24—5.28) 


Surveillance planning 


5.37 Transport Canada should clarify 
what information on industry and 
aviation companies should be used in 
making risk-based decisions, collect 
that information, assess its 
completeness and reliability, and 
develop risk profiles when preparing 


~ annual surveillance plans in the regions. 


(5.35-5.36) 


5.41 Transport Canada should clarify 
the minimum level of surveillance 
needed to provide the necessary 
coverage of civil aviation companies. 


(5.38-5.40) 


Agreed. By March 2013, the Department will have identified 
ways to accelerate follow-up on significant safety issues raised by 
stakeholders. The new process will continue to allow for 
consultation and regulatory changes. 


Agreed. Transport Canada is committed to maintaining 
Canada’s very strong civil aviation safety record. It has 
developed and begun implementation of a comprehensive action 
plan to guide the modernization of its structure and 
improvements to its systems, processes, and tools, and to ensure 
that resources are targeted to areas of highest risk. The Assistant 
Deputy Minister of Safety and Security is accountable for the 
implementation of this plan, and its progress is being monitored 
by a senior management steering committee chaired by the 
Associate Deputy Minister. Successful and complete 
implementation of the plan is a priority for the Department. 


By August 2012, the Department will complete and launch a 
standard risk assessment tool. The new tool is designed to collect 
information necessary to assess aviation companies and ensure 
consistency across all aviation enterprises, with regularly 
updated risk profiles to support a dynamic process of 
surveillance. These steps will improve the information base for 
the development of annual surveillance plans and overall 
surveillance activities. 


Agreed. By August 2012, Transport Canada will update and 
communicate to all inspectors and managers minimum 
surveillance levels for civil aviation companies. 
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Surveillance activities 


§.46 Transport Canada should review 
its staff instructions to clarify its 
expectations on the minimum 
documentation required and the level 
of management involvement expected 
in the oversight of surveillance 


activities. (5.42—5.45) 


§.50 ‘Transport Canada should 
determine why a number of inspections 
are falling short of plans and take 
corrective action. Deviations from 
original plans should be documented, 
and they should be approved by senior 
management. Senior management 
should also determine whether 
Transport Canada is still obtaining the 
desired level of assurance that 
companies are complying with air safety 


regulations. (5.47—5.49) 


§.59 Transport Canada should ensure 
that inspections are conducted and 
documented consistently according to 
established methodology. It should also 
consider whether existing 
documentation tools are sufficient or 
need to be redesigned or replaced. 


(5.51-5.58) 


9.63 ‘Transport Canada should ensure 
that all staff involved in inspections are 
trained in a timely manner so they can 
carry out their responsibilities. 


(5.60-5.62) 


Agreed. By December 2012, the Department will have 
completed an update of, and communicated to managers and 
staff clear direction on, minimum requirements for surveillance 
documentation and management involvement. 


Agreed. Minimum surveillance levels will be reflected in 
surveillance plans from 2012-13 forward. Performance will be 
monitored on a regular basis, any deviations from approved 
levels will require respective Regional Director General and 
Director General Civil Aviation approval nationally, and senior 
management will be provided with regular updates. 


Agreed. A review of tools and the launch of improved tools will 
be completed by December 2012. Guidance and new tools to 
improve documentation of inspections will be provided to staff 
as they are approved. This documentation will be reviewed 
regularly as part of Quality Assurance activities. 


Agreed. By July 2012, all current inspectors will have been given 
surveillance procedures training. There will be regular 
management reviews to ensure that mandatory training 
requirements continue to be met and that evolving training 
needs are addressed. 
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§.67 ‘Transport Canada should develop 
a clear expectation for appropriate 
management review and approval for 
planning, conducting, and reporting on 
surveillance activities to ensure that 
staff comply with established 
methodology. The nature, timing, and 
extent of that involvement should be 


documented. (5.64—5.66) 


Human resources planning 


5.74 ‘Transport Canada should identify 
the number of resources and the 
competencies it will need to plan and 
conduct inspections under its new 
surveillance approach, and develop a 
strategy to obtain them. (5.68—5.73) 


Quality assurance 


§.81 Transport Canada should develop 
a detailed quality assurance plan and 
make it a priority toimplementa _ 
rigorous quality assurance program that 
is consistent with best practices. 


(5.76—5.80) 
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Agreed. By June 2012, the Department will put in place a formal 
process to ensure appropriate management review and sign-off of 
surveillance reports. 


Agreed. The 2012-13 surveillance plan will be used to 
determine the number of resources and the competencies 
required by the Department. This information will inform 
updates to the national human resources plan for civil aviation, 
which will include strategies for recruitment and training. 


Agreed. The Department is working on a rigorous quality 
assurance plan that reflects updated surveillance and inspection 
methodology, which will be fully completed and implemented by 
March 2013. Results of quality assurance work will be used to 
improve the Department’s work related to its oversight 
responsibilities. 
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Importance of Crown corporations 


Special Examinations of Crown Corporations—2011 


61 This chapter presents the main points of special examination 
reports of Crown corporations that were issued to the corporations’ 
boards of directors between 1 January 2011 and 31 December 2011 
and that have subsequently been made public. These reports were 
transmitted throughout the year, and the respective systems and 
practices were examined during different time periods. 


6.2 Crown corporations are distinct legal entities whose names, 
mandates, powers, and objectives are set out in either the constituent 
legislation for the parent Crown corporation or the articles of 
incorporation under the Canada Business Corporations Act. There are 
two types of Crown corporations: those that are wholly owned directly 
by the Government of Canada (parent Crown corporations) and those 
that are wholly owned by other Crown corporations (subsidiaries). 


63 Crown corporations have more autonomy than most other 
government entities, in part, because they have commercial objectives 
as well as public policy objectives. They account for a significant 
portion of government activity and operate in many sectors of the 
Canadian economy, including transportation, energy, agriculture and 
fisheries, financial services, culture, and government services. 


64  Asof 31 December 2011, there were 46 parent Crown 
corporations and three subsidiaries that were directed to report as 
parent Crown corporations. These Crown corporations employed over 
100,000 people. More than 83 percent of these employees worked at 
four corporations: Canada Post Corporation, Canadian Broadcasting 
Corporation, Atomic Energy of Canada Limited, and VIA Rail Canada 
Inc. The vast majority of Crown corporations had fewer than 

3,000 employees. New Crown corporations are created periodically, 
but no new corporations were created in 2011. 


65 | Crown corporations manage more than $375 billion in assets. 
Note that owing to the unique nature of their operations, total asset 
holdings for the Bank of Canada, the Canada Pension Plan Investment 
Board, and the Public Sector Pension Investment Board have not been 
included in the Crown corporations’ total assets. Crown corporations 
fund their operations in a variety of ways. Some corporations are 
required by their enabling legislation to be financially self-sustaining 
and thus receive no parliamentary appropriations. Some are funded 
mainly through parliamentary appropriations. Others receive federal 
funding but also generate revenue. In the 2010-11 fiscal year, 

27 corporations received just over $6.7 billion in appropriations; 
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Role of the Office of 
the Auditor General 


Definition and objective of 
a special examination 


Significant deficiency—A major weakness ina 
Crown corporation’s key systems and practices 
that could prevent it from having reasonable 
assurance that its assets were safeguarded and 
controlled, its resources were managed 
efficiently and economically, or its operations 
were carried out effectively. 


82 percent of these appropriations went to five corporations: Canada 
Mortgage and Housing Corporation, Canadian Broadcasting 
Corporation, Atomic Energy of Canada Limited, Canadian Air 
Transport Security Authority, and VIA Rail Canada Inc. 


66 A board of directors or a similar governing body oversees the 
management of each corporation and holds management responsible 
for the corporation’s performance. The board of a parent Crown 
corporation is, in turn, accountable to Parliament through the 
responsible minister. 


67 Under Part X of the Financial Administration Act, the Auditor 
General is appointed to conduct annual audits of the financial 
statements and periodic special examinations of Crown corporations, 
either on his own or jointly with a private sector audit firm, unless he 
waives the appointment. 


6.8  Aspecial examination is an important accountability mechanism 
for Crown corporations. The objective of a special examination is to 
provide an independent opinion on whether there is reasonable 
assurance that a Crown corporation has systems and practices in place 
to ensure that its 


* assets are safeguarded and controlled; 


+ financial, human, and physical resources are managed economically 
and efficiently; and 


* operations are carried out effectively. 


6.9 Any major weakness in the key corporate systems and practices 
that could prevent a corporation from achieving these objectives is 
reported as a significant deficiency. When planning special 
examinations, we conduct a risk analysis to identify the systems and 
practices that we consider essential to providing the corporation with 
this assurance. We also establish criteria that we use to examine the 
corporation’s systems and practices. These criteria are based on our 
experience with performance auditing and our knowledge of the 
subject matter. They are selected in consultation with the corporation. 


6.10 The opinion we present in the special examination can take one 
of three forms. First, we may find there is reasonable assurance that no 
significant deficiencies exist in the systems and practices the 
corporation maintains to achieve the objectives noted above. 

Second, we may find there is one or more significant deficiencies. 
Finally, in rare situations, we may find there is no reasonable assurance 
that a corporation’s systems and practices achieve the objectives. 
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611 The frequency of a special examination, as determined by the 
Financial Administration Act, is at least once every 10 years. However, 
special examinations could be carried out more frequently, if required 
by the Governor in Council (the Cabinet acting in a legal capacity), 
the appropriate minister, the board of directors of the corporation, or 
the Auditor General. 


6.12 As part of the accountability mechanism for Crown corporations, 
the examiner reports on special examinations to the board of directors 
of a Crown corporation. Furthermore, the examiner may bring 
information from the special examination to the attention of the 
appropriate minister, after consulting with the board of directors. The 
examiner may also bring such information to Parliament’s attention, 
after consulting with the appropriate minister and the board of 
directors, by preparing a report for inclusion in the next annual report 
of the corporation. 


6.13 The Financial Administration Act requires that the board of 
directors submit all special examination reports to the appropriate 
minister and the President of the Treasury Board within 30 days of 
receiving them. The Act also requires that the board of directors make 
the reports available to the public within 60 days of receiving them. 


6.14. Reports were issued for the following three Crown corporations 
between | January 2011 and 31 December 2011, and have been 
subsequently made public: 


+ Canadian Dairy Commission, 
¢ Canadian Race Relations Foundation, and 


¢ Public Sector Pension Investment Board. 


615 No significant deficiencies were identified in these special 
examination reports. 


616 In addition, the special examinations highlight systems and 
practices that contribute to success, and they provide information and 
recommendations to boards of directors about other opportunities for 
improvement. For these three Crown corporations, our 
recommendations focused primarily on corporate governance, strategic 
planning, and risk management. Because of the small number of 
special examination reports issued during this period, it is difficult to 
identify overall trends. 


617 Since 2008, we have been reporting annually on the special 
examinations transmitted in each respective year. Cumulatively, we 
have reported on 29 special examinations of which 8 had significant 


Report of the Auditor General of Canada—Spring 2012 


Chapter 6 | 3 


SPECIAL EXAMINATIONS OF CROWN CORPORATIONS—201 1 


deficiencies and 18 did not. As noted earlier, these reports were 
transmitted to the respective boards of directors on different dates 
during each year, and the respective systems and practices were 
examined during different time periods. No follow-up audit work 
regarding any of the matters raised in these reports has been 
performed. 


6.18 The next section provides the main points from each of the 
three special examination reports. Full reports can be found on each 
corporation’s website. 
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Main Points of Special Examinations—2011 


Canadian Dairy Commission—Main Points 


[We issued the full report to the board of directors on 16 March 2011. 

The examination work covered the period from March 2009 to August 
2010. For the full report and our recommendations, please go to www.cdc.ca 
or contact the Commission. ] 


The Canadian Dairy Commission is a Crown corporation established 
in 1966 to coordinate the federal government’s and the provinces’ 
policies and roles in managing the dairy industry in Canada. Acting 
as both a facilitator and a stakeholder in various forums that influence 
Canadian dairy policy, the Commission supports the interests of all 
dairy stakeholders—producers, processors, exporters, consumers, 

and governments. 


The Commission has about 65 employees. The federal government 
funds approximately half of its administrative costs. Other costs, 
including marketing activities, are funded by dairy producers and 
the marketplace. 


The Commission reports to Parliament through the Minister of 
Agriculture and Agri-Food whom it advises on matters related to 
the dairy industry. 


We examined whether the Canadian Dairy Commission’s systems 

and practices provide it with reasonable assurance that its assets are 
safeguarded and controlled, its resources are managed economically and 
efficiently, and its operations are carried out effectively. Our examination 
focused on areas important to all Crown corporations, such as corporate 
governance, risk management, and strategic planning, and on areas of 
particular importance to the Commission, such as determining support 
prices and market-sharing quotas. Our examination covered the systems 
and practices that were in place between March 2009 and August 2010. 


In 2009, dairy production in Canada generated total net farm receipts 
of $5.5 billion and sales of $13.6 billion, representing 15 percent of the 
Canadian food and beverage sector, according to the Canadian Dairy 
Information Centre. The Canadian Dairy Commission plays a key role 
in ensuring that Canadians have a continuous supply of milk and other 
dairy products available and seeks to ensure that efficient producers 
have the opportunity to obtain a fair return on their labour 

and investment. 
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What we found 
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We found no significant deficiencies in the Commission’s systems and 
practices. A significant deficiency is reported when there is a major 
weakness in the Commission’s key systems and practices that could 
prevent it from having reasonable assurance that its assets are 
safeguarded and controlled, its resources are managed efficiently 

and economically, and its operations are carried out effectively. 


We noted good practices in a number of areas. We also noted some areas 
where the Commission would benefit from improving its practices. 


¢ The Commission has the key elements of an effective governance 
framework in place. The Board assumes stewardship of the 
Commission and works closely with management. Roles and 
responsibilities are clearly defined and understood, and the Board 
is supported by a simplified subcommittee structure. Some issues 
raised in our 2005 Special Examination Report have not been 
resolved—namely, it is difficult for the three-person Board to have 
the full range of skills needed for governance, and the Board does 
not have a process for directors to declare and manage conflicts 
of interest. 


¢ The Commission manages its key operational functions well. 
Its practices for determining support prices and market-sharing 
quotas are consistent with its legislation and objectives. It also has 
systems and practices in place for protecting its inventory of butter 
and skim milk powder and for issuing permits under the Special 
Milk Class Permit Program. 


¢ The Commission’s management of human resources provides it 
with the core competencies and skills it needs as a small Crown 
corporation. It has recognized the need for succession planning. 
The work to prepare for future retirements is ongoing. 


The Commission has responded. The Commission agrees with 
our recommendations. 
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Canadian Race Relations Foundation—Main Points 


[We issued the full report to the board of directors on 16 September 2011. 
The examination work covered the period from November 2010 to April 
2011. For the full report and our recommendations, please go to www.crt.ca 
or contact the Foundation. ] 


The Canadian Race Relations Foundation is a Crown corporation 
founded in November 1997. It was created as part of the 

1988 Japanese—Canadian Redress Agreement, which acknowledged 
that the treatment of Japanese—Canadians during and after World 
War II was unjust and violated principles of human rights. Under the 
terms of the Agreement, the federal government promised to create a 
Canadian Race Relations Foundation. 


The Foundation’s operating income is derived primarily from the 
income earned by investing a one-time $24 million endowment fund 
established by the Government of Canada when the Foundation was 
created. The Canadian Race Relations Foundation Act requires that the 
endowment itself be used solely for investment. 


The Foundation reports to Parliament through the Minister of 
Citizenship, Immigration and Multiculturalism. It is governed by a 
board of directors and has six employees; its office is in Toronto. The 
Foundation’s activities are mainly outreach and awareness activities, 
which are intended to contribute to the elimination of racism and 
racial discrimination. They include media events, use of social media 
tools, award, and education and training programs. 


We examined whether the Canadian Race Relations Foundation’s 
systems and practices provide it with reasonable assurance that its 
assets are safeguarded and controlled, its resources are managed 
economically and efficiently, and its operations are carried out 
effectively. In this first special examination of the Foundation, 

we focused on the areas of governance, strategic planning and 
risk management, investment management, and operations. 

Our examination covered the systems and practices that were in 
place between November 2010 and April 2011. 


The Foundation’s mandate, as specified in the Canadian Race Relations 
Foundation Act, is to facilitate the development, sharing, and 
application of knowledge and expertise throughout Canada in order 
to contribute to the elimination of racism and all forms of racial 
discrimination in Canadian society. The Foundation supports the 
federal government’s commitments and responsibilities under the 
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International Convention on the Elimination of All Forms of Racial 
Discrimination and under Canadian law for equal participation in 
Canadian society and the elimination of discrimination. 


What we found We found no significant deficiencies in the systems and practices of the 
Canadian Race Relations Foundation. We noted good practices in a 
number of areas, and we also noted some practices that the Foundation 
would benefit from improving. 


¢ The Foundation’s Board of Directors has most of the elements 
necessary for effective governance. The Board’s roles and 
responsibilities and those of the executive director are clearly 
defined, and the Board provides management with strategic 
direction and oversight. The Board has developed an effective 
working relationship with management, and has practices in place 
to maintain its independence. However, the Board competency 
profile has not been updated since March 2009, and that profile 
did not reflect the specific skills and experience required on the 
Board. In particular, the profile did not identify the extent of 
investment management expertise the Foundation needs to 
oversee the portfolio that provides the majority of its income. 


¢ The Foundation has most of the elements necessary to manage its 
investment portfolio. Its investment policy includes all the key 
elements necessary to guide the Foundation’s investment strategy, 
and the activities of three fund managers are overseen by an 
investment advisor and the Board. However, management and 
the Board need more complete performance information to better 
monitor the investment portfolio. 


¢ The Foundation’s strategic planning process sets clear direction 
for its operations. Planning of activities is mainly done through 
the development of the business plan, which aligns the activities 
with the Foundation’s strategic plan and mandate, and includes 
information that allows them to be monitored by management 
and the Board. The Foundation also identified its major 
organizational risks and developed action plans to mitigate them, 
which management started to implement. However, there has 
been no review of the actions taken or their effectiveness in 
mitigating the risks. 


The Foundation has responded. The Foundation agrees with all of 
the recommendations. 
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Public Sector Pension Investment Board—WMain Points 


[This examination was jointly completed with Deloitte & Touche LLP 

We issued the full report to the board of directors on 12 May 2011. 

The examination work covered the period from June 2010 to November 2010. 
For the full report and our recommendations, please go to www.investpsp.ca 
or contact the Corporation. ] 


The Public Sector Pension Investment Board (PSP Investments or the 
Corporation) is a Crown corporation created to invest the pension 
contributions (net of benefit payments made) of the Public Service, 
Canadian Forces, Royal Canadian Mounted Police, and Reserve Force 
pension plans (the Plans). Its mandate is to manage and invest these 
funds in the best interests of the contributors and beneficiaries, with a 
view to achieving a maximum rate of return, without undue risk of 
loss, having regard to the funding, policies and requirements of the 
pension plans. 


We examined whether PSP Investments’ systems and practices provide 
the Corporation with reasonable assurance that its assets are 
safeguarded and controlled, its resources are managed economically 
and efficiently, and its operations are carried out effectively. We 
focused on the areas of governance, risk management, strategic 
planning, performance measurement, investment management, human 
resource management, and information technology management. Our 
examination covered the systems and practices that were in place 


between June 2010 and November 2010. 


Net contributions from the plan members and sponsors which are 
transferred to PSP Investments represent an important portion of the 
assets which will serve to pay benefits of plan members upon their 
retirement. Ineffective systems and practices could lead to diminished 
returns and erosion of capital, potentially compromising the plans’ 
ability to meet their financial obligations. 


We found no significant deficiencies in the Corporation’s systems and 
practices. This means that the Corporation maintains systems and 
practices that provide it with reasonable assurance that its assets are 
safeguarded and controlled, its resources are managed economically 
and efficiently, and its operations are carried out effectively. A 
significant deficiency is reported when there is a major weakness in the 
Corporation’s key systems and practices that could prevent it from 
having that reasonable assurance. We noted good practices in most 
areas, as well as some areas where PSP Investments would benefit from 


certain improvements. 
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* The Corporation has the key elements of a strong governance 


framework, and its governance practices are consistent with 
industry practices for stewardship and oversight by boards of 
directors. However, the lack of the staggering of appointments to 
the Board of Directors, which is not under the control of PSP 


Investments, may potentially lead to significant turnover in the 
Board membership in 2014 and 2015. 


The Corporation’s risk management practices, particularly in the 
area of investment risk, provide for identification, monitoring, 
management, and reporting of risks to protect its assets from 
undue risk of loss. The Corporation continues to develop its risk 
measurement and risk management capabilities, in line with 
industry practice. 


The Corporation’s compensation framework and practices are 
comparable with those of the industry. Incentives for investment 
staff are designed to balance the need to attract and retain 
talented performers with the need to align behaviours with the 
Corporation’s investment strategies, policies, and risk tolerances. 
Staff engaged in risk and compliance functions are compensated 
in a manner that maintains their independence from the areas 
they oversee. The Board plays an active oversight role in the 
design and operation of compensation practices, and reviews and 
monitors them independently of management. 


The Corporation regularly benchmarks its practices against those 
of comparable organizations in the industry. Its practices in most 
areas are consistent with those of the industry. However, 
improvements can be made in public reporting of its Responsible 
Investment activities in order to better align with industry 
practices in this area. 


The Corporation has responded. The Corporation agrees with all 
the recommendations. 
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About the Chapter 


Objective 


The objective of this chapter was to bring to the attention of Parliament the results of the Office of the 
Auditor General’s special examination reports that were transmitted to the boards of directors of Crown 
corporations between | January 2011 and 31 December 2011. The Crown corporations have subsequently 
made these reports public. 


Scope and approach 


The chapter includes the main points of two special examinations where the Auditor General was the 
examiner, and one special examination where the Auditor General was the joint examiner. 


The approach to this chapter consisted of presenting information that was already public, providing 
information about what a special examination is, and presenting the main points of the three special 
examinations. 


Audit team 


Assistant Auditor General: Clyde M. MacLellan 
Principal: Marise Bédard 
Director: Raymond Murphy 


Caroline Bérubé 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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Auditor General of Canada 
Vérificateur général du Canada 


To the Honourable Speaker of the House of Commons: 


| have the honour to transmit herewith this 2012 Spring Report to the House of Commons, which is to be 
laid before the House in accordance with the provisions of subsection 7(5) of the Auditor General Act. 


Yours sincerely, 


ef, 


Michael Ferguson, FCA 


OTTAWA, 3 April 2012 
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Message from the Auditor General— 
Spring 2012 


I am pleased to present my first report to Parliament as Auditor 
General of Canada. 


This report addresses a wide range of issues that illustrate some of the 
challenges facing government today: 


* ensuring at the border that commercially imported consumer 
goods identified as potential health and safety risks conform with 
applicable Canadian laws and regulations; 


* acquiring major defence equipment through decisions that 
commit the government to a course of action well into the future; 


* managing Canada’s interest-bearing debt; 


* improving compliance with the Income Tax Act among those who 


are required to file a tax return or to register for GST/HST but 


Michael Ferguson, FCA : ; 
Auditor General of Canada who fail to do so; and 


* overseeing whether aviation companies are complying with 
the policies, regulations, and standards required for the safety 
of civil aviation within Canada’s borders. 


The sheer magnitude of operations in the programs we examined 


is impressive. For example: 


¢ The Canada Border Services Agency processed and 
released 13 million shipments of commercial products in 
the 2010-11 fiscal year. 


¢ National Defence has budgeted $9 billion to acquire 
65 F-35 fighter jets and $16 billion to operate and sustain 
them over 20 years. 


¢ The government’s interest-bearing debt managed by the 
Department of Finance Canada totalled $802 billion at 
31 March 2011; the interest charges that year, $30.9 billion, 
represented 11.4 percent of government expenses. 


* The Canada Revenue Agency’s Non-Filer/Non-Registrant 
program generated $2.8 billion in additional taxes, interest, and 
penalties assessed in each of the 2009-10 and 2010-11 fiscal 


years. 
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¢ Transport Canada oversees the safety practices of the second 
largest civil aviation aircraft fleet in the world, accounting for 
nearly three million flights in 2010. 


With areas of responsibility on such a large scale, governments must 
make choices in their day-to-day operations about which services to 
deliver and where to focus their activities to best serve the public good. 
In times of fiscal restraint, choices become tougher and trade-offs must 
be made. Some government organizations discussed in this report are 
managing this challenge; others need to improve. 


Border Controls on Commercial Imports 


Potentially unsafe products entering Canada without appropriate 
controls carry risks that could include illness or even death. 

The volume of imports into Canada is so large that it is not practical 
to apply controls at the border to every shipment that enters. In order 
to balance the requirement to examine shipments with the need to 
facilitate the free flow of goods, federal organizations responsible for 
product safety need to identify high-risk shipments on which to focus 
controls at the border. These controls include administering import 
requirements; targeting high-risk shipments, both automatically and 
through border lookouts; and examining intercepted shipments. 


We found that in most cases, the consumer goods in our audit that 
pose a risk to health and safety are adequately controlled at the border 
by the Canada Border Services Agency (CBSA) and the federal 
organizations responsible for them. However, some controls need to 
be better managed and the results better monitored by all the 
organizations involved, so they know whether they are directing their 
resources at the highest risks. 


Replacing Canada’s Fighter Jets 


This chapter looks at two separate but related activities in National 
Defence. One was its participation, starting in 1997, in the United 
States-led, international Joint Strike Fighter program to design, 
develop, and manufacture the F-35 Lightning II fighter aircraft. 
The other was the process to acquire the F-35 as the replacement 
for the fleet of CF-18 Hornet fighter jets. 


National Defence and Industry Canada recognized that Canada’s 
participation in the Joint Strike Fighter (JSF) program provided 
opportunities for industrial benefits to Canada, and they took 
appropriate steps to help Canadian industry take advantage of those 
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opportunities. In contrast, National Defence did not recognize early 
enough that its involvement in the program had procurement 
implications. Consequently, it did not engage Public Works and 
Government Services Canada (PWGSC) early enough to establish 
a suitable process, consistent with procurement rules and 

including appropriate safeguards, to manage a project of this 

nature and magnitude. 


When National Defence decided to recommend the acquisition of 
the F-35, it was too involved with the aircraft and the JSF Program to 
run a fair competition. It applied the rules for standard procurement 
projects but prepared key documents and took key steps out of proper 
sequence. As a result, the process was inefficient and not managed 
well. Key decisions were made without required approvals or 
supporting documentation. Information provided to decision makers 
was incomplete, and no plan was developed for extending the life of 
the CF-18 fleet in the event of prolonged delays in the delivery of a jet 
that is still being developed. 


For its part, PWGSC, in its role as the government’s procurement 
authority, did not do enough to ensure the integrity—the fairness 
and transparency—of the procurement process. 


National Defence did not follow the basics of good management that 
would be expected for a $25 billion commitment by the government. 
It is important that a purchase of this magnitude follow a rigorous, 
transparent process. 


Interest-Bearing Debt 


Public debt and the associated interest charges consume a large 
amount of financial resources. They affect the government's fiscal 
strength, limit policy choices, and influence what we can afford as a 
nation. The interest-bearing debt therefore needs to be managed and 
reported on properly. Clearer information allows for better policy 
debate, including debate on how to keep public finances sustainable. 


We found that the Department of Finance Canada has a sound 
decision-making system in place to support and develop effective 
market debt strategies. It has a risk management framework that allows 
it to respond to emerging risks and changes in funding requirements. 
However, the Department needs to improve its monitoring of and 
reporting on the overall performance of the debt management strategy 
to inform management decisions. And while Canada is a leader in 
reporting its public sector pension plan liabilities in its financial 


MESSAGE FROM THE AUDITOR-GENERAL—SPRING 2012 


statements, financial information on the pension plans is dispersed 
among several reports, making it difficult to readily understand the 
potential impact of these liabilities. 


Non-Filers and Non-Registrants—Canada Revenue Agency 


Income tax is the single most important source of government 
revenue. For the income tax system to operate fairly and effectively, 
all individuals and corporations need to file returns when the law 
requires it, and pay the taxes they owe. Not doing so reduces the 
amount of money available for important government programs such 
as health care, education, and environmental protection. Similarly, 
businesses that are required to register for GST or HST, but that fail 
to do so, may have a negative impact on government revenues. 


We found that overall, the Canada Revenue Agency’s Non-Filer/Non- 
Registrant program works, but there is room for improvement. The 
Agency has not integrated its research and past experience into its 
planning in order to improve the program. It has validated its process 
for selecting files to pursue, but not for rejecting files. It needs to 
determine the effectiveness of both in order to ensure that it is not 
focusing on files with little to no tax potential. 


Oversight of Civil Aviation—Transport Canada 


While Canada compares favourably with many other countries in 

its aviation safety record, any deterioration would significantly 

impact public confidence. This makes it critical that Transport Canada 
maintain a robust and effective regulatory framework for civil aviation 
safety. The Department’s civil aviation safety program puts the onus 
on large aviation companies to develop safety management systems 

in accordance with regulations, while the Department oversees 

their compliance. 


We found that Transport Canada has made real progress in its 

new approach to safety oversight. It has revised its surveillance 
methodology to be consistent with this approach, and its rigorous 
regulatory framework for civil aviation safety is consistent with 
international standards. However, the Department can take a long 
time to address emerging safety concerns—in some cases, more than 
10 years. We also found that the Department is not adequately managing 
the risks associated with its oversight of civil aviation. There are 
weaknesses in how it plans, conducts, and reports on its surveillance 
activities. For example, in the 2010-11 fiscal year, only two thirds of 
planned inspections were carried out—which is significant, since only 
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the higher-risk aviation companies are selected for inspection in any 
given year. In addition, Transport Canada has not yet identified how 
many inspectors and engineers it will need to oversee civil aviation— 
something it agreed to do in 2008 when we recommended it. 


Special Examinations of Crown Corporations—2011 


The final chapter in this report presents the main points of special 
examination reports on Crown corporations that were issued to the 
corporations’ boards of directors between 1 January 2011 and 

31 December 2011 and that the corporations subsequently made 
public. A special examination is an important accountability 
mechanism for Crown corporations. It provides an independent 
opinion on whether there is reasonable assurance that a Crown 
corporation has systems and practices in place to ensure that its assets 
are safeguarded and controlled; its financial, human, and physical 
resources are managed economically and efficiently; and its operations 
are carried out effectively. Any major weakness in the key corporate 
systems and practices that could prevent a corporation from achieving 
those objectives is reported as a significant deficiency. 


I am pleased to note that we found no significant deficiencies in 
our 2011 special examinations of the Canadian Dairy Commission, 
the Canadian Race Relations Foundation, and the Public Sector 
Pension Investment Board. 


The government’s decisions have significant consequences for the 
citizens of this country. In the current environment of managing with 
less, the challenge for public servants is to maximize the potential 
benefits of choices and trade-offs and to minimize negative 
consequences. 


On a personal note, I would like to acknowledge the efforts by staff 
of the Office of the Auditor General to make my introduction to the 
Office a smooth transition for everyone, even in the midst of our own 
operational review. I have joined a strong institution, with people who 
approach the work of the Office with dedication and professionalism. 
I look forward to working with them, with Members of Parliament, and 
with government officials to meet the challenges that Canada faces 


over the next decade. 
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Chapter 1 


What we examined 


Border Controls on Commercial 
Imports 


Main Points 


Under various acts and regulations, federal government organizations 
are responsible for ensuring the safety of commercially imported 
consumer goods that have been identified as a risk to the health and 
safety of Canadians. The Canada Border Services Agency (CBSA) is 
tasked with ensuring that commercially imported consumer goods 
enter Canada in conformity with applicable laws and regulations. 


To achieve this, the CBSA works with other federal government 
organizations to implement controls at the border. Controls at the 
border include administration of import requirements (for example, 
permits and authorizations), detection and interception of shipments 
targeted by federal organizations as high risk or non-compliant, 

and examination of selected shipments for admissibility into Canada. 
These controls are part of a broader product safety regime that also 
includes both pre- and post-border controls, such as licensing and 
market surveillance, and is supported by importers’ voluntary 
compliance. The regime is intended to control any high-risk products 
that are allowed to enter the country and thereby help ensure the 
safety of imported products in the marketplace. 


We examined the part of the product safety regime that is implemented 
at the border. Looking at selected commercially imported goods— 
including fertilizers, health products, pest control products, consumer 
products, consumer fireworks, vehicles, and tires—we examined how 
the CBSA, the Canadian Food Inspection Agency, Health Canada, 
Natural Resources Canada, and Transport Canada work together to 
ensure that products comply with applicable legislation when they 
enter Canada. 


Audit work for this chapter was substantially completed on 
30 September 2011. Further details on the conduct of the audit are 
in About the Audit at the end of this chapter. 
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Why it’s important 


Chapter 1 


What we found 


The volume of imports into Canada is so large that it is not practical 
for federal organizations to apply border controls to every shipment, 
nor is it necessary. According to the CBSA, it processed and released 
13 million shipments of commercial products in the 2010-11 fiscal year, 
about four million of which were subject to federal import requirements. 
Federal organizations must balance the requirement to examine 
shipments with facilitating the free flow of goods. To do this, they need 
to have systems and practices that can identify and concentrate on 
high-risk shipments of commercially imported products. 


Potentially unsafe products entering Canada without appropriate 
controls carry risks that could include illness and death. Even a less 
serious incident can have a significant effect on Canadian consumer 
confidence in imported products. 


- In most cases, imported consumer goods in our audit that pose a risk 
to the health and safety of Canadians are adequately controlled at 
the border by the Canada Border Services Agency, the Canadian 
Food Inspection Agency, Health Canada, Natural Resources 
Canada, and Transport Canada. With a few exceptions, the 
administration of import requirements and automatic targeting of 
high-risk shipments are working as intended. 


In the small percentage of cases where goods that did not meet 
import requirements were allowed to enter the country, most were 
products for which there was no agreement in place between Health 
Canada and the CBSA. While the CBSA has formal arrangements 
with the three other organizations in our audit, as yet it has no 
formal agreement with Health Canada that documents respective 
roles, responsibilities, policies, and procedures for implementing 
controls on several products under Health Canada’s responsibility, 
such as medical devices and pest control products. Until there is a 
formal agreement, border services officers do not have consistent 
instructions on procedures to follow for those products. 


Border lookouts and examinations are reserved for higher-risk 
shipments and they consume resources, yet their results are poorly 
documented. For example, examination results were recorded 
incorrectly or incompletely in 40 percent of cases we examined. 
Consequently, we could not determine whether the examinations 
are working as intended, nor can the organizations requesting them. 
Moreover, among the audited organizations there are gaps in the 
monitoring of all three border controls. For example, the CBSA’s 
target monitoring reports are inaccurate and incomplete. These gaps 
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make it difficult for federal organizations to know how well the 


controls are working and where resources and effort can be directed 
most effectively to manage risk. 


The entities have responded. The entities agree with all of 
the recommendations. Their detailed responses follow the 
recommendations throughout the chapter. 
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Chapter 2 


What we examined 


Chapter 2 


Replacing Canada’s Fighter Jets 


Main Points 


Canada currently operates a fleet of CF-18 Hornet fighter jets purchased 
in the 1980s with an original life expectancy until 2003, since extended 
to between 2017 and 2020 after a modernization program in early 2000. 
National Defence has determined that a suitable replacement for the 
CF-18 is required, or it will lose its ability to carry out domestic and 
international missions mandated by the Government of Canada. 


National Defence has been a partner in the Joint Strike Fighter (JSF) 
Program since 1997. Led by the United States, and with eight other 
country partners, the Program is undertaking concurrent design, 
development, and manufacturing of the F-35 Lightning II aircraft. It will 
eventually include a regime for long-term collaborative sustainment. 
Canada’s participation has been formalized by signing international 
memoranda of understanding—in 1997, 2002, and 2006—for each of 
the three major phases of the JSF Program. As of September 2011, the 
government had disbursed about CAN$335 million toward participation 
in the JSF Program and related support to Canadian industry. The 
government has committed a total of US$710 million to the Program. 


In May 2008, through the Canada First Defence Strategy, the federal 
government announced its intent to replace the CF-18 fleet with 

65 “next generation” fighter aircraft. Then, in July 2010, the 
government announced its decision to buy the F-35 Lightning II, 
without following a competitive process, as the CF-18 replacement. 


We examined whether National Defence, Industry Canada, and Public 
Works and Government Services Canada exercised due diligence in 
managing Canada’s participation in the JSF Program and in managing 
the federal decision-making process to acquire the F-35 as a 
replacement for the CF-18. 


Our conclusions relate only to the management practices and actions 
of public servants. We did not audit private sector contractors and, 
consequently, our conclusions do not pertain to the contractors’ 
practices or to their performance. We did not audit the merits of 

the F-35 aircraft. 
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Audit work for this chapter was substantially completed on 
30 September 2011. Further details on the conduct of the audit are 
in About the Audit at the end of this chapter. 


Why it’s important = Buying major defence equipment is subject to decision-making and 
project management processes whose aim is to ensure that decisions 
are well founded, projects are managed effectively, and goods and 
services are acquired in a way that enhances supplier access, 
competition, and fairness. National Defence, Industry Canada, 
Public Works and Government Services Canada (PWGSC), and 


central agencies are involved. 


Buying and maintaining the F-35, or any other fighter jet, will require a 
significant long-term financial commitment. The F-35 is still being 
developed and tested, and projections of purchase price and 
sustainment costs are still being refined. Decisions taken to date as well 
as those yet to come will have impacts for the next 40 years. 


What we found = + National Defence took the appropriate steps in managing Canada’s 
participation in the Joint Strike Fighter (JSF) Program to develop the 
F-35. National Defence engaged Industry Canada early, and together 
they managed industrial participation well. Early efforts to secure 
contract opportunities for Canadian companies were successful. 


- There were significant weaknesses in the decision-making process 
used by National Defence in acquiring the F-35 to replace the 
CF-18. By the end of 2006, the Department was actively involved in 
developing the F-35, and a number of activities had put in motion its 
eventual procurement. In the lead-up to the government’s 2010 
announcement, required documents were prepared and key steps 
were taken out of sequence. Key decisions were made without 
required approvals or supporting documentation. 


PWGSC did not fully carry out its role as the government’s 
procurement authority. Although it was not engaged by National 
Defence until late in the decision-making process, PWGSC endorsed 
the key decision to sole source the acquisition of the F-35 in the 
absence of required documentation and completed analyses. By that 
time, practically speaking, Canada was too involved with the aircraft 
and the JSF Program to run a fair competition. 


National Defence did not provide complete information in a timely 
manner. For example, briefing materials prepared for decision makers 
did not explain the basis for and limitations of projections of 
industrial benefits to Canadian companies, and the risks of relying on 
the projections for decision making. In addition, briefing materials 
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did not inform senior decision makers, central agencies, and the 
Minister of the problems and associated risks of relying on the F-35 
to replace the CF-18. Nor did National Defence provide complete 
cost information to parliamentarians. 


National Defence likely underestimated the full life-cycle costs of 

the F-35. The budgets for the F-35 acquisition (CAN$9 billion) and 
sustainment (CAN$16 billion) were initially established in 2008 
without the aid of complete cost and other information. Some of that 
information will not be available until years from now. If the budgets 
prove insufficient to cover total costs, the Department will have to 
find ways to cover additional costs that may be incurred. Alternatively, 
it may have to seek additional funds from the government or use funds 
from other parts of its capital or operating budgets. 


The departments have responded. National Defence agrees with our 
recommendation. Its response follows the recommendation. 


National Defence, Industry Canada, and Public Works and 
Government Services Canada have accepted the facts presented in the 
chapter. Both National Defence and Public Works and Government 
Services Canada disagree with the conclusions set out in paragraphs 


2.80 and 2.81. 
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What we examined 


Why it’s important 
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Interest-Bearing Debt 


Main Points 


Debt management refers to how the Government of Canada raises 
funds to meet its borrowing requirements, what it decides and does 
about the composition of the market debt, and how it governs these 
activities. The Government of Canada’s interest-bearing debt is made 
up of principally two sets of liabilities: market debt and public sector 
pension plan liabilities. Market debt is the part of the debt that the 
government borrows in financial markets and that is managed by the 
Department of Finance Canada. It totalled $597 billion at 

31 March 2011. Public sector pension plan liabilities are part of the 
non-market debt. These liabilities represent the government’s 
obligations to the employee pension plans of the public service, 
Canadian Forces, and Royal Canadian Mounted Police (RCMP). 
They totalled $146 billion at 31 March 2011. 


Together, market debt and the pension plan liabilities make up over 
92 percent of the interest-bearing debt, which totalled $802 billion at 
31 March 2011. In 2010-11, interest charges on the debt totalled 
$30.9 billion and represented 11.4 percent of government expenses. 


We examined how the Department of Finance Canada develops 
strategies to manage market debt. We looked at its risk management 
practices and at how it monitors and reports on performance of the 
debt-funding strategy. We also examined how the Department of 
Finance Canada and the Treasury Board of Canada Secretariat report 
information about charges on the interest-bearing debt as well as the 
budgetary impact of the public sector pension plan liabilities. 


Audit work for this chapter was substantially completed on 
31 October 2011. Further details on the conduct of the audit are 
in About the Audit at the end of this chapter. 


Debt management is the federal government's largest program (after 
transfers to seniors and to other levels of government). How market 
debt is managed has a direct impact on the government’s fiscal 
strength. Debt managers need to set appropriate market debt strategies 
using robust processes and tools—first, to balance costs and risks so 
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Chapter 3 


What we found 


that low-cost and stable funding can be raised, and second, to support 
well-functioning markets for Government of Canada securities. 


Deteriorating sovereign debt conditions in the Euro zone and in the 
United States show the importance of having sound debt strategies, as 
they provide stability and assurance to market participants. Sound debt 
strategies will also support the fiscal sustainability of public finances in 
the long term. 


In addition to market debt, public sector pension plan liabilities and 
associated debt charges have a large impact on the government’s fiscal 
situation. It is therefore important that Canadians be provided with 
clear information on that debt, the associated interest charges, and 
their impact on budget deficits or surpluses. The significant amounts 
involved can influence policy choices by limiting what the government 
can afford to do. 


« In developing its debt strategies, the Department of Finance Canada 
uses a sound process that relies on a detailed and robust debt strategy 
model and on the judgment of senior managers as well as on 
consultations with dealers and investors. The model and the 
qualitative/quantitative analyses help debt managers recommend 
preferred debt structures, and are major advances since our last audit 
12 years ago. 


The Department is monitoring and achieving the objective of a well- 
functioning market for Government of Canada securities. Until 
recently, it was using metrics that did not provide a full assessment of 
the extent to which it was achieving the objective of raising low-cost, 
stable funding for the Government of Canada by arriving at a desired 
debt structure that strikes a balance between costs and risks. 
Subsequent to the period under review, the Department started 
monitoring better the extent to which it balances costs and risks, but 
it cannot yet demonstrate that the debt strategy is achieving the low- 
cost, stable funding objective. While the Department publishes clear 
information on the market debt, it could publish more details to 
indicate why a particular debt strategy was chosen and how well the 
program is performing. 


The Department of Finance Canada has a sound risk management 
framework in place to assess and monitor emerging risks as well as 
changes in financial requirements that could affect the debt strategy. 
However, it does not promote Government of Canada securities to 
market participants as well as it could to appeal to a broader base of 
investors. 
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* Canada is a leader among member countries of the Organisation for 
Economic Co-operation and Development (OECD) in recognizing 
in its financial statements the obligations arising from public sector 
employee pension plans. In fact, very few other countries report 
these obligations on their financial statements. However, while 
complete financial information on the pension plans is available, it is 
dispersed among several reports and not presented in easy-to-read 
formats. It is therefore difficult for parliamentarians and Canadians 
to readily understand the potential impact of these liabilities on the 
budgetary balance and how they influence policy choices. In 
addition, the composition of projected interest charges reported in 
the Estimates is not clearly presented. Finally, the Department of 
Finance Canada does not have timely access to the quarterly updates 
from the Public Sector Pension Investment Board about actual 
returns on public sector pension investments, which would help the 
Department assess the impact that unforeseen fluctuations could 
have on budget surpluses or deficits. 


The entities have responded. The entities have agreed with our 
recommendations. Their detailed responses follow the 
recommendations throughout the chapter. 
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Non-Filers and Non-Registrants— 
Canada Revenue Agency 


Main Points 


The Canada Revenue Agency works to ensure that Canadians pay 
their required share of taxes and that the revenue base is protected. 
It is responsible for administering the Income Tax Act, which specifies 
when taxpayers are required to file a return. A non-filer is an 
individual, a corporation, or a trust who fails to file a tax return as 
required by legislation. 


Under the Excise Tax Act, businesses that meet certain criteria are 
required to register for the goods and services tax/harmonized sales tax 
(GST/HST). Non-registrants are businesses that fail to comply with 
this requirement. 


The Agency, through its Non-Filer/Non-Registrant (NF/NR) program, 
works to encourage individuals, corporations, and trusts to comply 
with the filing requirement and, in the case of businesses, with the 
GST/HST registration requirements. In the 2010-11 fiscal year, the 
NE/NR program’s salary budget was $39 million of the Agency’s total 
budget of $4.5 billion, and it employed 700 of the Agency’s 
approximately 39,000 employees. This relatively small program area 


generated $2.8 billion of additional taxes, interest, and penalties 
assessed in each of the 2009-10 and 2010-11 fiscal years. 


We looked at what the Agency has done to address non-compliance 
with filing and registration requirements. The audit focused on how 
the Agency identifies non-filers and non-registrants and how it plans, 
monitors, and reports on its actions to improve compliance. 


Audit work for this chapter was substantially completed on 
5 January 2012. Further details on the conduct of the audit are in 
About the Audit at the end of the chapter. 


Income tax is the single most important source of government 
revenue. For the income tax system to operate fairly and effectively, 

it is important that all individuals and corporations file returns when 
they are required by law to do so, and that they pay the taxes they owe. 
Not doing so reduces the amount of money available for important 
government programs such as health care, education, and the 
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environment. Similarly, businesses that are required to register for GST 
or HST, but do not, may have a negative impact on government 
revenues. The Agency has identified non-filers and non-registrants as 
high-priority issues to address, as part of its efforts to combat non- 
compliance, including the underground economy. 


What we found + Because the Agency does not have the resources to pursue all non- 
filers, it has developed a risk-scoring model to identify those it will 
pursue. Agency analysis indicated that the files it chooses to pursue 
result in returns filed and taxes assessed. However, it has not tested 
its screening to determine whether the files it chooses not to pursue 
from the initial 2.5 to 3 million files identified by the matching 
process should in fact be pursued. In addition, two thirds of the files 
initially selected for pursuit in the field are later determined to have 
low potential, and work on them is discontinued. In other words, the 
Agency does not know if its risk-scoring process is as effective as it 


could be. 


e 


The Agency uses identification projects to find taxpayers who may 
be participating in the underground economy. The majority of 
projects completed were meeting or exceeding their targets for 
number of tax returns filed and amounts assessed. 


The Agency’s planning process tends to repeat actions from one year 
to the next to identify non-filers and non-registrants. While it 
considers the cost of its processes to pursue these files when planning 
work, the Agency is not taking advantage of its research findings in 
its work plans in order to improve its results in achieving compliance. 


Current performance indicators focus on routine program 
activities—for example, tax dollars and the number of taxpayers 
assessed—rather than on the longer-term impact of the program, in 
particular the Agency’s success at getting non-compliant taxpayers 
to change their behaviour and file their returns. Various audits and 
program evaluations over the years have recommended action to 
develop additional performance measures and risk management of 
the Non-Filer/Non-Registrant (NF/NR) program and to improve its 
use of internal and external information. The Agency has made 
limited progress in implementing many of these recommendations. 


The Agency has responded. The Agency agrees with all of the 
recommendations. Its detailed responses follow the recommendations 


throughout the chapter. 
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Oversight of Civil Aviation— 
Transport Canada 


Main Points 


Aviation companies that want to operate commercially in Canada 
must obtain an authorization. In 2011, there were more than 

5,000 companies authorized to operate and more than 34,000 aircraft 
registered in Canada. To maintain their authorization to operate, these 
companies must meet the minimum safety standards required for the 
safe conduct of civil aviation. 


The minimum safety standards for aviation companies fall under the 
Aeronautics Act and the Canadian Aviation Regulations. The Act and 
the regulations form the main part of the regulatory framework for civil 
aviation. The overall responsibility for the maintenance of safe, regular, 
and efficient civil aviation operations, including the manufacturing 
and maintenance of aircraft, rests with the aviation industry. 


Transport Canada is responsible for developing and administering the 
policies, regulations, and standards required for the safe conduct of 
civil aviation within Canada’s borders. The Department is also 
responsible for overseeing whether aviation companies have complied 
with this safety framework, and for taking appropriate enforcement 
action where necessary. In 2009-10, Transport Canada spent over 
$148 million and dedicated about 1,400 employees to monitoring civil 
aviation across Canada. 


Our audit examined whether the Department has managed the risks 
associated with overseeing its civil aviation safety program. We focused 
on Transport Canada’s surveillance of air carriers, aircraft maintenance 
organizations, and airports in the National Airports System. 


Audit work for this chapter was substantially completed on 
30 September 2011. Further details on the conduct of the audit are in 
About the Audit at the end of this chapter. 


Transport Canada plays a key role in helping to ensure that Canada’s 
civil aviation safety framework meets minimum international safety 
standards. While Canada compares favourably with many other 
countries in its aviation safety record, any deterioration would 
significantly impact public confidence. This makes it critical that 
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‘Transport Canada maintain a robust and effective regulatory 
framework for civil aviation safety, especially since the International 
Civil Aviation Organization (ICAO) has projected a significant growth 
in aviation until 2025. 


Identifying aviation companies that present safety risks is a highly 
complex process that relies heavily on the judgment and experience of 
‘Transport Canada’s inspectors across Canada and on the information 
that is made available to them. 


The Department's inspectors cannot be continuously present in all 
aviation companies to assess their compliance with aviation safety 
standards. Consequently, Transport Canada must use risk management 
techniques to decide where, when, how often, and in how much depth 
it inspects aviation companies in order to obtain sufficient assurance 
that they are complying with Canada’s aviation safety requirements. 
If Transport Canada does not manage these surveillance risks well, 

it is unlikely to focus its scarce resources on aviation companies and 
operations that represent the highest risks. 


What we found + Transport Canada has developed a rigorous aviation safety regulatory 
framework that is consistent with standards established by the 
International Civil Aviation Organization, but it can take a long 
time to address emerging safety issues—in some cases, more than 
10 years. The Department has yet to fully implement a process that 
will address these issues more quickly. 


Since 2008, Transport Canada has made progress in evolving from 
the traditional surveillance approach—largely based on responding 
to regulatory requirements—to a systems-based approach designed 
for large and small aviation companies. This approach allows for 
more consistent and rigorous surveillance of aviation companies’ 


compliance with safety regulations. 


While some aspects of the new surveillance program are working 
well, there are weaknesses in critical areas. For example, information 
for assessing the risk indicators that Transport Canada uses to 
identify the high-risk aviation companies that should be inspected is 
not always available or kept up to date. A minimum acceptable level 
of surveillance has not been clearly established to indicate how long 
aviation companies can operate without being inspected, and only 
two thirds of planned inspections have been carried out. Most 
inspections are not fully conducted according to established 
methodology and are subject to little management oversight. 
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In addition, documentation of key decisions is weak. Finally, 
Transport Canada lacks a quality assurance program to continuously 
improve its surveillance program. 


Transport Canada recently developed a national human resources 
plan for the oversight of civil aviation to help ensure that it has the 
resources it needs to carry out its safety regulatory program. 
However, the plan does not specify the number of inspectors and 
engineers that are needed, although the Department agreed to 
provide these figures in its response to our 2008 recommendation. 
The Department has made progress with implementing key human 
resources strategies, but efforts to fully implement the new 
surveillance approach have been hampered by the lengthy 
reorganization and by resistance from some inspectors. 


The Department has responded. The Department agrees with all of 
our recommendations. Its detailed responses follow each 
recommendation throughout the chapter. 
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APPENDIX 


Appendix Report on the audit of the President of the Treasury Board's Anmua/ Report to Parliament on 
the Tabling of Crown Corporations’ Reports 2011 


Tablings in Parliament for parent Crown corporations: Annual reports and summaries of corporate plans and budgets 


Section 152 of the Financial Administration Act (the Act) requires the President of the Treasury Board to 
lay before each House of Parliament, a report on the timing of the tabling, by appropriate ministers, of 
annual reports and summaries of corporate plans and of budgets of Crown corporations. This report must 


be tabled by 31 December. 


The Act requires the Auditor General of Canada to audit the accuracy of the President of the Treasury 
Board’s report on the timing of tabling and to present the results in his annual report to the House of 
Commons. 


The President of the Treasury Board’s Annual Report to Parliament on the Tabling of Crown Corporations’ 
Reports 2011 was tabled on 9 December 2011. 


At the time that our Fall 2011 report was going to print, we were unable to include the results of the above 
audit, since the President of the Treasury Board’s report had not yet been finalized. Our auditor’s report 
was subsequently appended to the President’s report and is reproduced in this report to Parliament. 
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Auditors’ Report 


To the House of Commons: 


As required by subsection 152(2) of the Financial Administration Act, I have audited, for the year ended 
31 July 2011, the information contained in the Annual Report to Parliament on the Tabling of Crown 
Corporations’ Reports 2011. The information contained in the report is the responsibility of the President of 
the Treasury Board. My responsibility is to express an opinion on the information contained in the report 
based on my audit. 


I conducted my audit in accordance with the standards for assurance engagements established by the 
Canadian Institute of Chartered Accountants. Those standards require that I plan and perform an audit 
to obtain reasonable assurance as to whether the information contained in the report is free of significant 
misstatement. My audit included examining, on a test basis, evidence supporting the dates and other 
disclosures provided in the report. 


In my opinion, the information contained in the Annual Report to Parliament on the Tabling of Crown 
Corporations’ Reports 2011 is accurate, in all significant respects, with the section “Deadlines for tabling 
in Parliament” contained within the report. 


Lucie Cardinal, CA 
Principal 
for the Interim Auditor General of Canada 


2 November 2011 
Ottawa, Canada 
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